Caching is one of the most important techniques used to build high-performance and scalable systems. Instead of repeatedly querying slow databases, applications store frequently accessed data in a fast in-memory store like Redis, dramatically improving response time and reducing database load.

Redis has become one of the most widely used technologies in modern backend architectures because of its speed, simplicity, and powerful data structures. From session storage and API caching to rate limiting and real-time analytics, Redis plays a key role in handling large-scale traffic efficiently.

This guide explains the core concepts of caching and Redis, including caching strategies, Redis data structures, essential commands, eviction policies, and common production patterns used in scalable systems. Whether you're learning backend engineering or preparing for system design interviews, understanding Redis and caching is a must-have skill.

1. Caching Fundamentals

What is Caching

Caching is storing a copy of frequently accessed data in a faster storage layer so future requests can be served without going back to the original, slower source. The core idea is simple: if you already computed or fetched something expensive, save it temporarily so the next request gets it instantly.

Why Caching is Needed

Every system eventually hits a wall where the database becomes the bottleneck. When thousands of users hit the same endpoint simultaneously, querying the database for every request is wasteful and slow. Caching solves this by absorbing repeated reads.

Problems caching directly addresses:

• Database latency: A typical PostgreSQL query takes 20-100ms. Redis responds in under 1ms. For read-heavy workloads, that difference is enormous.
• High traffic spikes: A viral post, a flash sale, a live event — these create sudden traffic bursts that databases struggle to handle. A cache absorbs the load.
• Repeated identical queries: Fetching the same product details, user profile, or config values thousands of times per second makes no sense. Cache it once, serve it many times.

Cache Hit vs Cache Miss

• Cache hit: The requested data exists in the cache. Serve it directly. Fast, cheap, ideal.
• Cache miss: The data is not in the cache. Fall through to the database, fetch it, optionally store it in the cache for next time.

Hit rate is the key metric. A 95% hit rate means only 5% of requests touch the database. In high-traffic systems, even a 1% improvement in hit rate can significantly reduce database load.

Cache vs Database

Where Caching is Used in Real Systems

• User session data
• Product catalog and pricing
• Homepage feed or trending content
• Auth tokens and permissions
• Rate limit counters
• Search results
• Computed aggregations (dashboards, leaderboards)

2. Types of Caching

Client-Side Caching

The browser or mobile app stores responses locally. HTTP cache headers (Cache-Control, ETag, Expires) control this. Good for static assets, infrequently changing API responses.

Example: A user's browser caches a product image for 24 hours. No network request is made on repeated visits.

CDN Caching

Content Delivery Networks cache content at edge locations geographically close to users. CDNs like Cloudflare, Fastly, and AWS CloudFront cache HTML pages, images, JS/CSS bundles.

Example: A news article cached at 200 edge locations globally. A user in Mumbai gets content from the nearest edge node, not from the origin server in the US.

Application-Level Caching

The application server maintains a cache in-process (local memory) or via an external cache like Redis. This is where most backend engineering decisions happen.

Example: An Express.js or Django app caching database query results in Redis before returning them to the client.

Database Caching

Databases have internal caches (buffer pools, query caches). MySQL's InnoDB buffer pool caches frequently accessed pages. This is mostly automatic and not directly controlled by application engineers.

Distributed Caching

When you run multiple application servers, a single in-process cache does not work — each server has its own memory. A distributed cache like Redis sits outside all app servers as a shared cache that every server reads from and writes to.

Example: A fleet of 20 API servers all reading from the same Redis cluster. Consistent cache state across all nodes.

3. Introduction to Redis

What Redis Is

Redis (Remote Dictionary Server) is an open-source, in-memory data structure store. It functions as a cache, database, and message broker. Unlike a simple key-value store, Redis supports rich data structures: strings, hashes, lists, sets, sorted sets, streams, and more.

Created by Salvatore Sanfilippo in 2009, Redis became the default caching layer for production systems at companies like Twitter, GitHub, Stack Overflow, and Airbnb.

Why Redis Became the Industry Standard

• Extremely fast: sub-millisecond response times at scale
• Simple mental model: keys map to values, values can be complex structures
• Atomic operations: counters, queues, leaderboards without race conditions
• Built-in expiration: TTLs make cache management straightforward
• Persistence options: optional durability for more than just ephemeral caching
• Cluster support: horizontal scaling built into the server

Redis vs Traditional Databases

4. Why Redis is Fast

Redis achieves exceptional performance through a combination of architectural decisions:

• In-memory storage: All data lives in RAM. No disk seeks, no I/O waits. RAM access is orders of magnitude faster than disk.
• Single-threaded event loop: Redis processes one command at a time using a non-blocking I/O event loop (similar to Node.js). No lock contention, no thread context switching.
• Efficient data structures: Redis data structures are carefully optimized in C. A sorted set uses a skip list + hash map combo for O(log N) operations.
• Minimal network overhead: Redis uses a simple text-based protocol (RESP). Commands are compact and parsed fast.
• No disk I/O on reads/writes: By default, reads and writes happen entirely in memory. Persistence writes (RDB/AOF) happen asynchronously and don't block command processing.

Redis Request Processing Model

Client sends command
    --> Kernel receives data (TCP)
    --> Redis event loop picks it up
    --> Command parsed and executed in memory
    --> Response written back to client

The entire cycle happens in microseconds. There is no thread pool, no connection thread per client — just a single loop handling all I/O via epoll/kqueue.

5. Redis Architecture

Core Components

• Redis Server: The process that holds all data in memory and handles all commands
• Redis Client: Any application connecting over TCP (redis-cli, application libraries like ioredis, Jedis, redis-py)
• Event Loop: Single-threaded loop that handles all client connections, reads commands, executes them, sends responses
• In-memory store: The actual data — organized as a hash table of key-value pairs at the top level
• Persistence layer: Optional background processes writing RDB snapshots or AOF logs to disk

6. Redis Installation and Setup

Installing Redis

# Ubuntu/Debian
sudo apt install redis-server

# macOS
brew install redis

# Docker
docker run -d -p 6379:6379 redis

Running and Connecting

# Start Redis server
redis-server

# Connect via CLI
redis-cli

# Test connection
PING
# Returns: PONG

# Connect to remote host
redis-cli -h 127.0.0.1 -p 6379 -a yourpassword

Basic Configuration (redis.conf)

bind 127.0.0.1
port 6379
maxmemory 256mb
maxmemory-policy allkeys-lru
requirepass yourpassword

7. Redis Key Concepts

Keys and Values

Every piece of data in Redis has a key (always a string) and a value (can be one of several data types). Keys are case-sensitive. There is no implicit hierarchy — namespacing is a naming convention only.

Key Naming Conventions

Use colon-separated namespaces. This keeps keys readable and avoids collisions across different parts of your application.

user:profile:123
product:price:567
order:details:9812
session:token:abc123
rate:limit:user:456

TTL (Time to Live)

Every key can have an expiration time. Once expired, Redis automatically deletes the key. This is how you prevent stale data from accumulating.

SET user:session:123 "data"
EXPIRE user:session:123 3600        # expires in 1 hour

# Or set with TTL in one command
SET user:session:123 "data" EX 3600

# Check remaining TTL
TTL user:session:123
# Returns: seconds remaining, -1 (no expiry), -2 (key doesn't exist)

8. Redis Data Structures

Strings

The most basic type. Can hold text, numbers, serialized JSON, or binary data up to 512MB.

When to use: Session tokens, counters, simple cached values, feature flags.

SET user:name "Alice"
GET user:name

# Atomic counter (no race condition)
INCR page:views
DECR inventory:item:42

# Append
APPEND log:entry "new line"

Real-world example: Storing a serialized JSON object for a user profile. Fast reads, no joins needed.

Hashes

A hash maps field names to values inside a single key. Think of it as a row in a database table — one key, multiple attributes.

When to use: Storing objects with multiple attributes. More memory-efficient than storing each field as a separate key.

HSET user:100 name "Alice" age 25 city "Mumbai"
HGET user:100 name
HGETALL user:100
HDEL user:100 city
HINCRBY user:100 age 1

Real-world example: User profile data where you need to update individual fields without rewriting the whole object.

Lists

An ordered list of strings. Elements can be added or removed from both ends. Internally a linked list.

When to use: Task queues, recent activity feeds, message queues, job queues.

LPUSH tasks "task1"         # add to front
RPUSH tasks "task2"         # add to back
LPOP tasks                  # remove from front
LRANGE tasks 0 -1           # get all elements
LLEN tasks                  # count elements

Real-world example: A background job queue. Workers LPOP from the list. Producers RPUSH new jobs.

Trade-off: LRANGE on large lists is O(N). Avoid storing unbounded lists without trimming via LTRIM.

Sets

An unordered collection of unique strings. Duplicate inserts are silently ignored.

When to use: Tracking unique visitors, tags, relationships (followers/following), deduplication.

SADD tags "redis" "cache" "backend"
SMEMBERS tags
SISMEMBER tags "redis"      # check membership
SCARD tags                  # count members

# Set operations
SUNION tags1 tags2          # union
SINTER tags1 tags2          # intersection
SDIFF tags1 tags2           # difference

Real-world example: Storing user IDs who have seen a notification. SISMEMBER in O(1) to check if a user already saw it.

Sorted Sets

Like sets, but every member has a score (floating point). Members are always sorted by score. Allows range queries by rank or score.

When to use: Leaderboards, priority queues, rate limiting sliding windows, time-series data indexed by timestamp.

ZADD leaderboard 100 "player1"
ZADD leaderboard 250 "player2"
ZADD leaderboard 75  "player3"

ZRANGE leaderboard 0 -1 WITHSCORES   # ascending
ZREVRANGE leaderboard 0 2             # top 3
ZRANK leaderboard "player1"           # rank of player
ZINCRBY leaderboard 50 "player1"      # add to score

Real-world example: A gaming leaderboard updated in real-time. ZREVRANGE gives you the top N players instantly.

Bitmaps

Not a separate type — Bitmaps are strings where individual bits are addressed. Extremely memory-efficient for binary data at scale.

When to use: Tracking daily active users, feature flags per user ID, presence tracking.

SETBIT user:active:2024-01-15 1001 1   # user 1001 was active on Jan 15
GETBIT user:active:2024-01-15 1001
BITCOUNT user:active:2024-01-15        # count active users that day

Real-world example: 100 million users tracked for daily activity costs only 12MB in bitmap form.

HyperLogLog

A probabilistic data structure for counting unique elements with very low memory usage. Trades a small error margin (~0.81%) for massive memory savings.

When to use: Counting unique visitors, unique search queries, approximate cardinality at scale.

PFADD unique:visitors "user1" "user2" "user3"
PFCOUNT unique:visitors      # approximate unique count
PFMERGE combined key1 key2   # merge two HyperLogLogs

Real-world example: Counting unique page views. Storing 1 billion unique IDs exactly would take gigabytes. HyperLogLog does it in 12KB.

Streams

An append-only log data structure. Supports consumer groups for distributed message processing.

When to use: Event sourcing, message queues, activity logs, real-time data pipelines.

XADD mystream * event "order_created" order_id "456"
XREAD STREAMS mystream 0
XLEN mystream

9. Core Redis Commands Reference

String Commands

SET key value
SET key value EX 3600        # with expiry
GET key
MSET k1 v1 k2 v2             # multi-set
MGET k1 k2                   # multi-get
INCR counter
INCRBY counter 5
DECR counter
DEL key
EXISTS key

Hash Commands

HSET user:1 name "Alice" age 25
HGET user:1 name
HMGET user:1 name age
HGETALL user:1
HDEL user:1 age
HKEYS user:1
HVALS user:1
HINCRBY user:1 age 1

List Commands

LPUSH queue "task1"
RPUSH queue "task2"
LPOP queue
RPOP queue
LRANGE queue 0 -1
LLEN queue
LTRIM queue 0 99             # keep only first 100 elements
BLPOP queue 30               # blocking pop, wait 30 seconds

Set Commands

SADD users "user1" "user2"
SMEMBERS users
SISMEMBER users "user1"
SCARD users
SREM users "user1"
SUNION set1 set2
SINTER set1 set2

Sorted Set Commands

ZADD leaderboard 100 "player1"
ZRANGE leaderboard 0 -1
ZREVRANGE leaderboard 0 -1
ZRANK leaderboard "player1"
ZREVRANK leaderboard "player1"
ZSCORE leaderboard "player1"
ZINCRBY leaderboard 10 "player1"
ZRANGEBYSCORE leaderboard 50 200

10. Caching Strategies

Cache Aside (Lazy Loading)

The application manages the cache. On read, check cache first. On miss, fetch from DB and write to cache. On write, update the DB and invalidate or update the cache.

How it works:

1. Read request comes in
2. Check Redis — if hit, return data
3. If miss, query database
4. Write result to Redis with TTL
5. Return data to client

# Pseudocode flow
value = redis.get("user:100")
if value is None:
    value = db.query("SELECT * FROM users WHERE id = 100")
    redis.set("user:100", value, ex=3600)
return value

When to use: Read-heavy workloads, data that can tolerate brief staleness, general-purpose caching.

Pros: Only caches what is actually requested, resilient to cache failures.

Cons: First request always misses (cache cold start), potential for stale data if DB updates don't invalidate cache.

Write Through

On every write to the database, simultaneously write to the cache. Cache is always up to date.

How it works:

1. Application writes data
2. Write to Redis AND database in the same operation
3. Reads always hit the cache

When to use: Data that is written and then immediately read back, systems where stale cache is unacceptable.

Pros: Cache is always fresh, no stale reads.

Cons: Every write hits both systems — higher write latency, cache fills with data that may never be read.

Write Behind (Write Back)

Write to cache immediately. Write to database asynchronously (batched or delayed).

How it works:

1. Application writes to Redis only
2. Background worker flushes cache changes to DB periodically

When to use: Write-heavy workloads where write latency is critical (e.g., analytics counters, click tracking).

Pros: Very low write latency, write batching reduces DB load.

Cons: Risk of data loss if cache crashes before flush. Complex to implement correctly.

Refresh Ahead

Cache proactively refreshes entries before they expire, based on predicted access patterns.

How it works: Before a key's TTL hits zero, a background job refetches and re-caches the value.

When to use: Predictable access patterns, content that must never go stale (e.g., homepage data, config values).

Pros: No cache misses for hot data.

Cons: Wastes resources refreshing data that might never be requested again.

11. Cache Eviction Policies

When Redis reaches its maxmemory limit, it needs to evict keys. The policy controls which keys get removed.

Production recommendation: allkeys-lru for pure caches. volatile-lru when Redis serves both cache and persistent storage roles.

# Set in redis.conf
maxmemory 512mb
maxmemory-policy allkeys-lru

12. Redis Persistence

Redis is in-memory, but it offers two persistence mechanisms for durability.

RDB (Redis Database Snapshots)

Periodic point-in-time snapshots of the entire dataset written to disk. Redis forks a child process to write the dump.

# In redis.conf
save 900 1       # save if at least 1 key changed in 900 seconds
save 300 10      # save if 10 keys changed in 300 seconds
save 60 10000    # save if 10000 keys changed in 60 seconds

# Manual snapshot
BGSAVE

AOF (Append Only File)

Logs every write command to a file. On restart, Redis replays the log to reconstruct state.

# In redis.conf
appendonly yes
appendfsync everysec    # flush to disk every second (balanced)
# appendfsync always    # flush every command (safest, slowest)
# appendfsync no        # let OS decide (fastest, least safe)

RDB vs AOF

Production recommendation: Use both. RDB for fast restarts and backups, AOF for durability.

13. Redis Replication

Redis supports master-replica (formerly master-slave) replication. The replica receives a full copy of data from the master and then streams incremental updates.

How Replication Works

1. Replica connects to master and sends REPLICAOF command
2. Master sends full RDB snapshot to replica
3. Replica loads snapshot, then receives and applies ongoing write commands
4. Replicas serve read requests, master handles all writes

# On replica node
REPLICAOF master-host 6379

# Check replication status
INFO replication

When to Use Replication

• Scale read traffic across multiple replicas
• Provide a hot standby for failover
• Offload analytics queries to replicas

Important: Replication is asynchronous by default. A replica may lag slightly behind the master. For critical reads, always read from the master.

14. Redis Cluster

Redis Cluster provides horizontal scaling by sharding data across multiple nodes.

How It Works

• Redis divides the key space into 16,384 hash slots
• Each master node owns a range of hash slots
• A key's slot is computed as: CRC16(key) % 16384
• Clients are redirected to the correct node for each key

# Minimal 3-master cluster
redis-cli --cluster create \\
  127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 \\
  --cluster-replicas 1

# Check cluster info
CLUSTER INFO
CLUSTER NODES

Key Considerations

• Keys that need to go to the same slot use hash tags: {user:100}:profile and {user:100}:settings land on the same node
• Multi-key commands (MGET, MSET) only work if all keys hash to the same slot
• Minimum recommended setup: 3 masters with 3 replicas for high availability

15. Redis Pub/Sub

Redis Pub/Sub implements the publish-subscribe messaging pattern. Publishers send messages to channels. Subscribers receive all messages on channels they are subscribed to. No message persistence — if a subscriber is offline, it misses the message.

# Terminal 1 - Subscriber
SUBSCRIBE notifications
SUBSCRIBE channel1 channel2   # multiple channels
PSUBSCRIBE news.*             # pattern subscription

# Terminal 2 - Publisher
PUBLISH notifications "User logged in"
PUBLISH news.sports "Match started"

When to Use

• Real-time notifications (in-app alerts, push triggers)
• Live dashboard updates
• Chat systems (with the understanding that messages are not persisted)
• Fanout events where multiple services need to react to the same event

When Not to Use

Pub/Sub has no message persistence and no delivery guarantees. If your subscriber process restarts, messages sent while it was down are lost. For reliable messaging, use Redis Streams instead.

16. Redis Streams

Redis Streams is a persistent, append-only log. Unlike Pub/Sub, messages are stored and can be replayed. Supports consumer groups for distributed processing.

# Add to stream
XADD orders * event "order_created" order_id "789" user_id "123"

# Read from stream
XREAD COUNT 10 STREAMS orders 0

# Read only new messages
XREAD BLOCK 0 STREAMS orders $

# Consumer groups
XGROUP CREATE orders processors $ MKSTREAM
XREADGROUP GROUP processors worker1 COUNT 5 STREAMS orders >
XACK orders processors <message-id>

# Stream info
XLEN orders
XINFO STREAM orders

When to Use

• Event-driven architectures where events need to be replayed or reprocessed
• Background job queues with at-least-once delivery
• Activity feeds, audit logs, order pipelines
• Multiple consumers processing the same stream independently

17. Rate Limiting with Redis

Redis is the standard tool for implementing rate limiting due to its atomic operations and TTL support.

Fixed Window

Simplest approach. Count requests per user per time window.

# Increment counter for this user this minute
INCR rate:user:123:2024011510     # key includes the current minute
EXPIRE rate:user:123:2024011510 60

# Check before incrementing
GET rate:user:123:2024011510
# If count >= limit, reject request

Trade-off: Allows burst at window boundary. A user could make 100 requests at 11:59 and 100 at 12:00 — 200 requests in 2 seconds.

Sliding Window (with Sorted Sets)

More accurate. Track exact timestamps of requests using a sorted set.

# Add current request with timestamp as score
ZADD rate:user:123 1705312800 "request-uuid-1"

# Remove entries older than 60 seconds
ZREMRANGEBYSCORE rate:user:123 0 (now - 60)

# Count requests in window
ZCARD rate:user:123
# If count >= limit, reject

Token Bucket with Lua Script

For production, use Lua scripts for atomic token bucket logic. This prevents race conditions between check and increment.

local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local current = redis.call('INCR', key)
if current == 1 then
    redis.call('EXPIRE', key, 60)
end
if current > capacity then
    return 0
end
return 1

18. Redis in System Design

Typical Caching Architecture

Client Request
    --> Load Balancer
    --> API Server
        --> Check Redis Cache
            --> Cache Hit: return data immediately (<1ms)
            --> Cache Miss: query Database (20-100ms)
                --> Store result in Redis
                --> Return data to client

Latency Impact

With a 90% cache hit rate, average response time drops from 30ms to 3ms — a 10x improvement.

Database Load Reduction

If 95% of reads are served from cache, the database handles 5% of the original traffic. This extends database capacity, reduces costs, and improves write throughput since fewer reads compete for DB connections.

19. Real-World Redis Use Cases

Session Storage

Store authenticated user sessions in Redis with a TTL matching session timeout. Much faster than database session lookups on every request.

SET session:abc123 '{"user_id": 100, "role": "admin"}' EX 86400
GET session:abc123
DEL session:abc123     # on logout

Leaderboards

Sorted sets are purpose-built for this. Scores update atomically. Range queries return top-N players in O(log N + M).

ZINCRBY game:leaderboard 50 "player:999"
ZREVRANGE game:leaderboard 0 9 WITHSCORES    # top 10

Shopping Cart

Store cart items as a hash per user. Fast reads, partial updates without rewriting the full cart.

HSET cart:user:100 product:42 2 product:17 1
HGETALL cart:user:100
HDEL cart:user:100 product:17

Real-Time Analytics

Count events atomically. Bitmaps for unique users. HyperLogLog for approximate counts at scale.

INCR analytics:page:home:views
PFADD analytics:unique:visitors:today "user:100" "user:200"
PFCOUNT analytics:unique:visitors:today

API Rate Limiting

As described in section 16 — Redis is the standard tool across the industry.

Message Queues

Lists as simple queues. BLPOP for blocking consumers.

RPUSH email:queue '{"to":"a@example.com","subject":"Welcome"}'
BLPOP email:queue 0    # worker blocks until job arrives

20. Common Cache Problems

Cache Stampede (Thundering Herd)

What happens: A popular cache key expires. Hundreds of concurrent requests all miss at the same time, all query the database simultaneously, all try to write to Redis simultaneously.

Real-world scenario: A trending product page expires at midnight. 10,000 users simultaneously trigger database queries.

Mitigation:

• Mutex/lock: Only one request fetches from DB. Others wait for the cache to be populated.
• Probabilistic early expiration: Start refreshing before TTL expires using a small random chance as TTL approaches zero.
• Background refresh: An async worker refreshes hot keys before they expire.

# Mutex pattern
SET cache:lock:product:42 1 NX EX 5    # NX = only set if not exists
# If SET succeeds, this request fetches from DB and populates cache
# Others check lock, wait, then read from cache

Cache Penetration

What happens: Requests for data that doesn't exist in the database or cache keep bypassing the cache and hitting the database. Common in scraping or malicious traffic patterns.

Real-world scenario: Attacker queries user:0, user:-1, user:999999999 — none exist in DB, cache never populates, database gets hammered.

Mitigation:

• Cache null results with a short TTL: SET user:0 "NULL" EX 60
• Bloom filter: Maintain a probabilistic set of valid keys. Reject requests for keys not in the filter before they reach the cache or database.

Cache Avalanche

What happens: A large number of cache keys expire at the same time. All traffic suddenly hits the database simultaneously.

Real-world scenario: You preloaded 10,000 product cache entries at startup with the same 1-hour TTL. At exactly T+1 hour, all 10,000 keys expire simultaneously.

Mitigation:

• Jitter on TTL: Add a random offset to expiration times: EX = base_ttl + random(0, 300) — entries expire spread over a window, not all at once.
• Persistent hot keys: Never expire the most critical cache entries, refresh them via background jobs.
• Circuit breaker: If database error rate spikes, temporarily serve stale cache data.

21. Redis vs Memcached

When to choose Memcached: You only need simple string caching, you need raw multi-threaded throughput, and you have no need for persistence, pub/sub, or complex data structures. In practice, most teams choose Redis because its additional capabilities rarely come with a meaningful cost.

22. Redis Best Practices

Key Design

• Always namespace keys using a predictable format like service:entity:id
• Keep keys short but descriptive — long keys waste memory
• Avoid spaces and special characters in key names
• Document key patterns just like database schemas

Examples:

user:profile:123
order:details:9812
product:inventory:42
session:token:abc123

Good key design prevents collisions and makes debugging production systems easier.

TTL Discipline

• Always set a TTL for cached data when possible
• Avoid storing temporary cache entries without expiration
• Add small randomness to TTL values to prevent mass expiration

Example:

SET product:42'{
	"name":"Laptop",
	"price":1200
}' EX3600

Adding slight TTL variation helps avoid cache avalanche situations.

Avoid Large Values

Large values increase memory usage and network overhead.

Instead of storing one huge object:

user:123-> large JSON object

Split the data logically:

user:profile:123
user:settings:123
user:preferences:123

This improves partial updates and reduces serialization overhead.

Monitor Redis

Always monitor Redis in production.

Important metrics include:

• Memory usage
• Cache hit rate
• Evicted keys
• Connected clients
• Command latency

Useful commands:

INFO memory
INFO stats
SLOWLOG GET
MONITOR

Monitoring helps detect memory pressure, slow commands, and traffic spikes.

Avoid Blocking Commands

Some commands can block the Redis server if the dataset is large.

Avoid:

KEYS *
LRANGE huge_list0-1
SMEMBERS large_set

Prefer incremental scanning:

SCAN
SSCAN
HSCAN
ZSCAN

These iterate through data gradually without blocking the server.

Quick Reference

Conclusion

Redis is one of the most powerful tools used in modern backend systems to improve performance and scalability. By storing frequently accessed data in memory, it significantly reduces database load and improves application response times. Beyond simple caching, Redis provides rich data structures that enable features like leaderboards, rate limiting, queues, and real-time analytics.

Understanding caching strategies, eviction policies, and common cache failure scenarios is essential to use Redis effectively in production systems. When designed correctly, Redis can handle massive traffic spikes while keeping systems stable and responsive.

Mastering Redis and caching concepts is a valuable skill for backend engineers and is frequently tested in system design interviews as well as used heavily in real-world distributed systems.

If you found this cheatsheet useful, consider sharing it with your friends or colleagues who are learning backend development or preparing for system design interviews.

A Relational Database Management System (RDBMS) powers modern applications by organizing data into structured tables, enforcing relationships with keys, and ensuring reliable, consistent transactions using SQL and ACID principles.

1. RDBMS Fundamentals

What is an RDBMS?

A Relational Database Management System is software that stores data in structured tables and enforces relationships between them. The "relational" part comes from relational algebra — data is organized so you can query across tables using shared keys.

Examples: PostgreSQL, MySQL, Oracle, SQL Server, SQLite.

Why RDBMS? (Real-world context)

Most business data is naturally relational. An order belongs to a customer. A product belongs to a category. RDBMS lets you model that reality directly, enforce rules at the database level, and query across entities without duplicating data.

• Enforces data integrity (you can't have an order with no customer)
• Supports complex queries without application-level joins
• ACID compliance makes it safe for financial, medical, and transactional workloads
• Decades of tooling, optimization, and community support

RDBMS vs DBMS

Mental model: DBMS is a filing cabinet. RDBMS is a filing cabinet with enforced cross-references between folders, a query engine, and a transaction log.

2. Core Concepts

Tables, Rows, Columns

• Table — a named set of data with a fixed structure (like a spreadsheet with rules)
• Row (tuple/record) — one instance of data in a table
• Column (attribute/field) — a named property with a defined data type

Every column has a data type (INT, VARCHAR, DATE, BOOLEAN, etc.). This isn't just for storage — the database uses types to validate, index, and compare data correctly.

Schema

A schema is the blueprint. It defines what tables exist, what columns each has, what types they are, and what constraints apply. Think of it as the contract between your application and your data.

CREATE TABLE users (
  id       INT PRIMARY KEY,
  email    VARCHAR(255) UNIQUE NOT NULL,
  name     VARCHAR(100) NOT NULL,
  created_at TIMESTAMP DEFAULT NOW()
);

Primary Key

• Uniquely identifies each row in a table
• Cannot be NULL
• Should be immutable — once set, never changed
• Usually a surrogate key (auto-increment INT or UUID) rather than a natural key (email, phone)

Why surrogate keys? Natural keys change. Someone's email changes. Their phone number changes. An auto-increment ID never does.

Foreign Key

• A column (or set of columns) in one table that references the primary key of another
• Enforces referential integrity — you can't reference a row that doesn't exist

CREATE TABLE orders (
  id         INT PRIMARY KEY,
  user_id    INT NOT NULL,
  total      DECIMAL(10,2),
  FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);

ON DELETE CASCADE means: if the user is deleted, delete their orders too. Alternatives: SET NULL, RESTRICT (block the delete). Choose carefully — wrong cascade behavior is a data loss nightmare.

Constraints

Constraints live in the database, not in your application code. This is intentional — application bugs can be patched, but corrupt data is permanent.

3. Keys in Detail

Common confusion points:

• Every primary key is a candidate key, but not every candidate key is the primary key
• A table can have multiple candidate keys (e.g., id, email, phone) but only one primary key
• A foreign key's value doesn't have to be unique — many orders can reference the same user
• Composite keys are common in junction tables (user_id + role_id together are unique)

Interview trap: "Can a foreign key reference a non-primary key?" Yes — it can reference any UNIQUE column, not just the PK. Most people assume it must be PK.

4. Relationships

One-to-One (1:1)

A user has one profile. A country has one capital. Rare in practice. Usually signals you could merge the tables — but sometimes you split for performance (keeping heavy columns separate) or security (separating sensitive data).

CREATE TABLE user_profiles (
  user_id    INT PRIMARY KEY,
  bio        TEXT,
  avatar_url VARCHAR(500),
  FOREIGN KEY (user_id) REFERENCES users(id)
);

One-to-Many (1:N)

The most common relationship. One customer, many orders. One author, many posts. The foreign key lives on the "many" side.

-- orders.user_id references users.id
-- One user -> many orders

Many-to-Many (M:N)

Students take many courses. Courses have many students. You can't express this with just two tables — you need a junction (bridge/associative) table.

CREATE TABLE student_courses (
  student_id INT NOT NULL,
  course_id  INT NOT NULL,
  enrolled_at DATE,
  PRIMARY KEY (student_id, course_id),
  FOREIGN KEY (student_id) REFERENCES students(id),
  FOREIGN KEY (course_id) REFERENCES courses(id)
);

The junction table's PK is composite (student_id + course_id). You can also add extra columns to it — enrollment date, grade, status.

Entity Relationships:

5. SQL Fundamentals

SQL has four sub-languages. Know them cold — interviewers ask this.

-- DDL
CREATE TABLE products (id INT PRIMARY KEY, name VARCHAR(100), price DECIMAL(8,2));
ALTER TABLE products ADD COLUMN stock INT DEFAULT 0;
DROP TABLE products;

-- DML
INSERT INTO products (name, price) VALUES ('Keyboard', 49.99);
UPDATE products SET price = 44.99 WHERE id = 1;
DELETE FROM products WHERE stock = 0;

-- DQL
SELECT name, price FROM products WHERE price < 100 ORDER BY price DESC;

-- TCL
BEGIN;
  UPDATE accounts SET balance = balance - 500 WHERE id = 1;
  UPDATE accounts SET balance = balance + 500 WHERE id = 2;
COMMIT;

TRUNCATE vs DELETE: TRUNCATE removes all rows without logging individual deletions — much faster, but cannot be rolled back in most databases and does not fire triggers. Use DELETE when you need control or auditability.

6. CRUD Operations

-- CREATE
INSERT INTO users (name, email) VALUES ('Priya Sharma', 'priya@example.com');

-- READ with filtering, sorting, pagination
SELECT id, name, email
FROM users
WHERE created_at >= '2024-01-01'
ORDER BY name ASC
LIMIT 20 OFFSET 40;

-- UPDATE (always use WHERE — a bare UPDATE touches every row)
UPDATE users SET email = 'new@example.com' WHERE id = 7;

-- DELETE
DELETE FROM users WHERE id = 7;

-- Conditional insert (upsert)
INSERT INTO users (id, name, email)
VALUES (7, 'Priya', 'priya@example.com')
ON CONFLICT (id) DO UPDATE SET email = EXCLUDED.email;

Common mistake: Running UPDATE or DELETE without a WHERE clause in production. Always test with a SELECT first using the same WHERE condition.

7. Joins

The heart of relational databases. A join combines rows from two or more tables based on a related column.

Mental model: Think of two spreadsheets. A join is asking "show me rows where these two sheets have matching values in these columns."

-- INNER JOIN: orders with their customer names
SELECT o.id, u.name, o.total
FROM orders o
INNER JOIN users u ON o.user_id = u.id;

-- LEFT JOIN: all users, even those with no orders
SELECT u.name, COUNT(o.id) AS order_count
FROM users u
LEFT JOIN orders o ON o.user_id = u.id
GROUP BY u.id;

-- SELF JOIN: employees and their managers (same table)
SELECT e.name AS employee, m.name AS manager
FROM employees e
LEFT JOIN employees m ON e.manager_id = m.id;

Interview trap: "What's the difference between LEFT JOIN and LEFT OUTER JOIN?" They're the same thing. OUTER is optional syntax.

Join Types:

8. Aggregation

Aggregates collapse multiple rows into a single result. Used constantly in reporting.

SELECT
  category,
  COUNT(*)          AS total_products,
  AVG(price)        AS avg_price,
  MAX(price)        AS max_price,
  SUM(stock * price) AS inventory_value
FROM products
GROUP BY category
HAVING COUNT(*) > 5
ORDER BY inventory_value DESC;

WHERE vs HAVING:

• WHERE filters rows before grouping
• HAVING filters groups after aggregation
• You cannot use aggregate functions in WHERE

-- Wrong
SELECT category, COUNT(*) FROM products WHERE COUNT(*) > 5 GROUP BY category;

-- Right
SELECT category, COUNT(*) FROM products GROUP BY category HAVING COUNT(*) > 5;

9. Advanced Queries

Subqueries

A query inside another query. Used when you need the result of one query to drive another.

-- Find users who have placed at least one order
SELECT name FROM users
WHERE id IN (SELECT DISTINCT user_id FROM orders);

-- Find products more expensive than the average
SELECT name, price FROM products
WHERE price > (SELECT AVG(price) FROM products);

Correlated Subqueries

The inner query references the outer query. Executes once per row — can be slow on large tables.

-- For each user, find their most recent order date
SELECT u.name,
  (SELECT MAX(o.created_at) FROM orders o WHERE o.user_id = u.id) AS last_order
FROM users u;

When performance matters, rewrite correlated subqueries as JOINs or CTEs.

Window Functions

Window functions compute a value across a "window" of related rows without collapsing them (unlike GROUP BY).

SELECT
  name,
  department,
  salary,
  RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS rank_in_dept,
  AVG(salary) OVER (PARTITION BY department) AS dept_avg
FROM employees;

Common window functions: RANK(), ROW_NUMBER(), DENSE_RANK(), LAG(), LEAD(), SUM() OVER, AVG() OVER.

Mental model: GROUP BY collapses rows. Window functions keep all rows and add a computed column.

10. Normalization

Normalization is the process of organizing data to reduce redundancy and improve integrity. Each normal form solves a specific class of problem.

The Problem Without Normalization

Problems: Alice's email appears twice — update one, forget the other. Product price duplicated everywhere.

1NF — First Normal Form

Rule: Each column must hold atomic (indivisible) values. No repeating groups.

Violation: Storing "Math, Science, English" in a single subjects column.

Fix: One subject per row, or a separate subjects table.

2NF — Second Normal Form

Rule: Must be in 1NF + no partial dependencies (non-key column depends on only part of a composite PK).

Violation: In order_items(order_id, product_id, product_name) — product_name depends only on product_id, not the full PK.

Fix: Move product_name to a products table.

3NF — Third Normal Form

Rule: Must be in 2NF + no transitive dependencies (non-key column depends on another non-key column).

Violation: employees(id, dept_id, dept_name) — dept_name depends on dept_id, not on id.

Fix: Move dept_name to a departments table.

BCNF — Boyce-Codd Normal Form

Stricter version of 3NF. Every determinant must be a candidate key. In practice, 3NF is sufficient for most systems.

When to Denormalize

• High-read, low-write reporting tables
• Data warehouses and analytics (star schema, fact tables)
• When join cost exceeds storage cost at scale
• Caching pre-computed aggregates for dashboards

Denormalization is a deliberate performance trade-off, not laziness. The mistake is denormalizing before you've proven you need to.

Normalization Flow Diagram:

11. Indexing

What and Why

An index is a separate data structure that lets the database find rows without scanning every row in the table. Without an index, every query is a full table scan — O(n). With a B-tree index, lookup is O(log n).

Analogy: A database table is a book. An index is the book's index at the back — instead of reading every page, you go straight to page 247.

How it Works (B-Tree)

The default index type. A balanced tree where leaf nodes hold the actual row pointers (or values for covering indexes). Each lookup traverses from root to leaf — typically 3–4 levels for millions of rows.

Types of Indexes

CREATE INDEX idx_users_email ON users(email);
CREATE INDEX idx_orders_user_date ON orders(user_id, created_at);
CREATE UNIQUE INDEX idx_users_phone ON users(phone);
-- Partial index: only index active users
CREATE INDEX idx_active_users ON users(email) WHERE status = 'active';

Composite Index: Left-Prefix Rule

An index on (a, b, c) can serve queries filtering on a, a+b, or a+b+c. It cannot serve queries filtering only on b or c.

-- Index on (user_id, created_at)
SELECT * FROM orders WHERE user_id = 5 AND created_at > '2024-01-01'; -- uses index
SELECT * FROM orders WHERE user_id = 5;                                -- uses index
SELECT * FROM orders WHERE created_at > '2024-01-01';                  -- does NOT use index

Trade-offs

• Indexes speed up reads but slow down writes (INSERT/UPDATE/DELETE must update the index too)
• Each index consumes disk space
• Too many indexes on write-heavy tables is a common performance killer
• Rule of thumb: index columns you frequently filter, join, or sort by

12. Transactions

What is a Transaction?

A transaction is a unit of work that either completes entirely or doesn't happen at all. It groups multiple operations into an all-or-nothing block.

BEGIN;
  UPDATE accounts SET balance = balance - 1000 WHERE id = 1;
  UPDATE accounts SET balance = balance + 1000 WHERE id = 2;
COMMIT;
-- If anything fails between BEGIN and COMMIT, ROLLBACK undoes both updates

ACID Properties

Transaction Flow Diagram:

SAVEPOINT

BEGIN;
  INSERT INTO orders ...;
  SAVEPOINT after_order;
  INSERT INTO payments ...;  -- if this fails
  ROLLBACK TO after_order;   -- undo only the payment, keep the order
COMMIT;

13. Concurrency Control

Why it's Needed

Multiple users hitting the database simultaneously create race conditions. Without control, two transactions can read the same data, modify it, and overwrite each other.

Locking

• Shared Lock (S): Read lock. Multiple transactions can hold it simultaneously.
• Exclusive Lock (X): Write lock. Only one transaction; blocks everyone else.
• Row-level lock: Only locks the affected row (preferred, more concurrency)
• Table-level lock: Locks the entire table (fast to acquire, kills concurrency)

Deadlock

Transaction A holds lock on row 1, wants row 2.

Transaction B holds lock on row 2, wants row 1.

Both wait forever.

Databases detect deadlocks and kill one transaction (the "victim"). Your application must handle this by retrying.

Prevention: always acquire locks in the same order across transactions.

Isolation Levels

• Dirty Read: Reading uncommitted data from another transaction
• Non-Repeatable Read: Re-reading a row gets different data (another tx updated it)
• Phantom Read: Re-running a query gets different rows (another tx inserted/deleted)

Higher isolation = more safety, but more lock contention and lower throughput.

14. Database Optimization

Query Optimization Tips

• SELECT only the columns you need — SELECT * forces the DB to fetch all columns
• Filter early with WHERE before joining when possible
• Use indexes on JOIN columns and WHERE clauses
• Avoid functions on indexed columns in WHERE: WHERE YEAR(created_at) = 2024 doesn't use the index; WHERE created_at >= '2024-01-01' AND created_at < '2025-01-01' does
• Avoid SELECT DISTINCT unless truly needed — it adds a sort/dedup step
• Use EXISTS instead of IN with large subquery results
• Paginate with keyset pagination (WHERE id > last_seen_id LIMIT 20) instead of OFFSET for large tables

EXPLAIN / EXPLAIN ANALYZE

EXPLAIN ANALYZE
SELECT * FROM orders WHERE user_id = 5 AND created_at > '2024-01-01';

Key things to look for:

• Seq Scan — full table scan, usually bad on large tables
• Index Scan — using an index, good
• Nested Loop vs Hash Join — matters at scale
• rows= estimate vs actual rows — large mismatch means stale statistics (run ANALYZE)
• cost= — planner's estimate; focus on actual time in ANALYZE output

Common Mistakes

• No index on foreign key columns (causes full scan on every JOIN)
• Using LIKE '%term%' on large tables (can't use B-tree index)
• N+1 query problem: fetching 100 posts, then 1 query per post for author
• Not using connection pooling in production
• Running migrations without testing on production-sized data

15. Advanced Features

Views

A stored query that acts like a virtual table. Doesn't store data itself.

CREATE VIEW active_users AS
SELECT id, name, email FROM users WHERE status = 'active';

SELECT * FROM active_users; -- runs the underlying query

Use views for: simplifying complex queries, hiding sensitive columns, creating stable interfaces over evolving schemas.

Don't use views for: performance optimization (a view doesn't cache data — it re-runs each time). Use materialized views for caching.

Stored Procedures

Precompiled SQL logic stored in the database. Can accept parameters, contain control flow, and return results.

CREATE PROCEDURE transfer_funds(from_id INT, to_id INT, amount DECIMAL)
BEGIN
  START TRANSACTION;
    UPDATE accounts SET balance = balance - amount WHERE id = from_id;
    UPDATE accounts SET balance = balance + amount WHERE id = to_id;
  COMMIT;
END;

CALL transfer_funds(1, 2, 500.00);

Trade-off: logic in the database is harder to version, test, and deploy. Prefer application-layer logic unless there's a specific reason (performance, shared logic across multiple apps).

Triggers

Automatically execute logic in response to INSERT, UPDATE, or DELETE.

CREATE TRIGGER log_price_change
AFTER UPDATE ON products
FOR EACH ROW
WHEN (OLD.price <> NEW.price)
INSERT INTO price_audit (product_id, old_price, new_price, changed_at)
VALUES (NEW.id, OLD.price, NEW.price, NOW());

Use triggers for: audit logging, denormalized counter updates.

Avoid triggers for: business logic — they're invisible, hard to debug, and cause surprises.

Cursors

Row-by-row processing within a stored procedure. Almost always the wrong choice — set-based SQL operations are orders of magnitude faster. Use cursors only when row-by-row processing is genuinely unavoidable (rare ETL edge cases).

16. Scaling Databases

Vertical vs Horizontal Scaling

Replication

One primary (write) database, one or more replicas (read copies). Changes on primary are streamed to replicas.

• Read replicas absorb read traffic (reporting, analytics, API reads)
• Replication lag is real — reads from replicas may be slightly stale
• If your app reads immediately after writing, use the primary for that read

Sharding

Splitting data across multiple database instances based on a shard key. User IDs 1–1M on shard 1, 1M–2M on shard 2.

Trade-offs:

• Cross-shard joins are expensive or impossible — design schemas to avoid them
• Choosing the wrong shard key causes hot spots (one shard gets all the traffic)
• Resharding as you grow is painful

Most applications shouldn't shard until they're at massive scale. It's a last resort, not a first step.

Scaling Diagram:

17. SQL vs NoSQL

Choose SQL when: strong consistency matters, data is relational, you need complex queries, compliance requires auditability.

Choose NoSQL when: massive write throughput (Cassandra), flexible schema (MongoDB for CMS), pure key-value cache (Redis), graph relationships (Neo4j).

Interview trap: "Is NoSQL always faster?" No. PostgreSQL with proper indexes beats many NoSQL solutions for typical workloads. The choice is about data model fit and scale patterns, not raw speed.

18. Real-World System Design

E-Commerce Database

users       (id, name, email, password_hash, created_at)
addresses   (id, user_id, street, city, country, is_default)
categories  (id, name, parent_id)  -- self-referential for hierarchy
products    (id, name, description, price, stock, category_id, seller_id)
orders      (id, user_id, address_id, status, total, created_at)
order_items (id, order_id, product_id, quantity, unit_price)
payments    (id, order_id, method, status, amount, processed_at)
reviews     (id, user_id, product_id, rating, body, created_at)

Key design decisions:

• order_items.unit_price stores price at time of purchase — product price can change later
• categories.parent_id self-references for nested categories (Electronics > Phones > Android)
• orders.status uses an ENUM or varchar with CHECK constraint (pending, paid, shipped, delivered, cancelled)
• Index on products(category_id), orders(user_id), order_items(order_id)

Blog System

users      (id, username, email, bio, avatar_url)
posts      (id, author_id, title, slug, body, status, published_at, created_at)
tags       (id, name, slug)
post_tags  (post_id, tag_id)  -- junction table
comments   (id, post_id, user_id, parent_id, body, created_at)

Key design decisions:

• posts.slug is UNIQUE and indexed — used in URLs instead of exposing numeric IDs
• comments.parent_id self-references for threaded replies
• posts.status ENUM: draft, published, archived
• Full-text index on posts(title, body) for search

Banking System

customers     (id, name, national_id, email, kyc_status)
accounts      (id, customer_id, type, balance, currency, status, created_at)
transactions  (id, from_account_id, to_account_id, amount, type, status, created_at, reference_id)
audit_log     (id, table_name, row_id, action, changed_by, changed_at, old_values, new_values)

Key design decisions:

• Never update balance directly. Balance is derived from transactions, or updated only inside a transaction with locking
• transactions is append-only — never update or delete transaction rows
• reference_id is idempotency key — prevents duplicate charges if client retries
• Use SERIALIZABLE isolation for transfers or optimistic locking with a version column
• audit_log uses a trigger on accounts and transactions — every change is recorded

-- Safe transfer with locking
BEGIN;
  SELECT balance FROM accounts WHERE id = 1 FOR UPDATE;  -- exclusive row lock
  SELECT balance FROM accounts WHERE id = 2 FOR UPDATE;
  UPDATE accounts SET balance = balance - 500 WHERE id = 1;
  UPDATE accounts SET balance = balance + 500 WHERE id = 2;
  INSERT INTO transactions (from_account_id, to_account_id, amount, type, status)
    VALUES (1, 2, 500, 'transfer', 'completed');
COMMIT;

19. Quick Revision Summary

Fundamentals

• RDBMS stores data in tables with enforced relationships via keys
• Primary key: unique + not null + immutable. Foreign key: references PK in another table
• Constraints live in the DB, not just the app

SQL

• DDL (schema), DML (data changes), DQL (queries), TCL (transactions)
• WHERE filters rows, HAVING filters groups, you can't use aggregates in WHERE
• Joins combine tables: INNER (matches only), LEFT (all left), FULL (all rows)
• Window functions keep all rows + add computed column, unlike GROUP BY

Normalization

• 1NF: atomic values. 2NF: no partial deps. 3NF: no transitive deps
• Denormalize for performance only after proving it's needed

Indexing

• B-tree default, left-prefix rule for composite indexes
• Indexes slow writes — don't over-index write-heavy tables
• Covering index: query fetches everything from the index, never touches the table

Transactions

• ACID: Atomicity, Consistency, Isolation, Durability
• Higher isolation = safer, but more contention
• Deadlocks: always acquire locks in the same order

Scaling

• Vertical first, then replicas for reads, then sharding as last resort
• Replication has lag — don't read from replica immediately after writing

Conclusion

This RDBMS cheatsheet covers everything from core fundamentals to real-world system design, giving you a solid foundation for interviews and practical development. Focus on understanding the concepts, not just memorizing them—because real strength comes from applying these ideas in real systems.

If you found this helpful, consider sharing it with your friends, developers, or teammates—it might help someone else prepare smarter and faster.

A complete and practical guide to Low Level Design (LLD), covering OOP principles, SOLID, design patterns, class relationships, and real-world system design problems. This cheatsheet is designed to help backend developers and interview candidates build clean, scalable, and maintainable code with confidence.

1. LLD Fundamentals

What is LLD

Low Level Design is the process of defining the internal structure of individual components in a system. It answers the question: how will each module be built, not just what it will do.

While High Level Design (HLD) tells you that a system has a "Notification Service," LLD tells you what classes exist inside it, how they communicate, what patterns they use, and how edge cases are handled.

In interviews, LLD tests your ability to translate requirements into clean, extensible, object-oriented code. In real systems, good LLD is what separates a codebase that survives scaling from one that collapses under its own weight.

LLD vs HLD

Importance in Interviews and Real Systems

In FAANG and product-based interviews, the LLD round evaluates:

• Whether you can identify the right abstractions
• Whether your code is extensible without modification
• Whether you understand design patterns and know when NOT to use them
• Whether your classes have clear, single responsibilities

In real systems, poor LLD leads to:

• God classes that nobody wants to touch
• Tight coupling that makes testing a nightmare
• Impossible-to-extend code whenever requirements change

Clean Code Principles

These are not optional niceties. They are the baseline expectation.

• Readability: Code is read far more than it is written. Name things clearly. A method called processData() tells you nothing. calculateInvoiceTotal() tells you everything.
• Maintainability: Can a new engineer fix a bug without understanding the entire codebase? If not, your design has failed.
• Extensibility: Can you add a new payment method without touching existing payment code? Good LLD makes this possible.

2. Object-Oriented Programming (OOP)

Class and Object

A class is a blueprint. An object is a live instance of that blueprint with its own state.

public class BankAccount {
    private String accountId;
    private double balance;

    public BankAccount(String accountId, double initialBalance) {
        this.accountId = accountId;
        this.balance = initialBalance;
    }

    public double getBalance() {
        return balance;
    }
}

// Object
BankAccount account = new BankAccount("ACC001", 5000.0);

Common mistake: Treating classes as data bags with no behavior. If your class only has getters/setters and no meaningful methods, it is an anemic domain model - a known anti-pattern.

Encapsulation

Hide internal state. Expose only what is necessary through a controlled interface.

Why it matters: If balance is public, any part of the system can directly set it to -99999. Encapsulation prevents this.

// BAD
public class Order {
    public double total; // Anyone can mutate this
}

// GOOD
public class Order {
    private double total;

    public void addItem(double price) {
        if (price <= 0) throw new IllegalArgumentException("Price must be positive");
        this.total += price;
    }

    public double getTotal() {
        return total;
    }
}

Abstraction

Expose what something does, hide how it does it. Achieved through interfaces and abstract classes.

public interface PaymentGateway {
    boolean processPayment(double amount, String currency);
}

public class StripeGateway implements PaymentGateway {
    @Override
    public boolean processPayment(double amount, String currency) {
        // Stripe-specific HTTP calls, token handling, etc.
        return true;
    }
}

The caller only knows about PaymentGateway. It does not care whether it is Stripe, Razorpay, or PayPal underneath.

Inheritance

A subclass inherits state and behavior from a parent class. Use it when a genuine "is-a" relationship exists.

public class Vehicle {
    protected String brand;
    protected int speed;

    public void accelerate(int amount) {
        this.speed += amount;
    }
}

public class Car extends Vehicle {
    private int numberOfDoors;

    public Car(String brand, int numberOfDoors) {
        this.brand = brand;
        this.numberOfDoors = numberOfDoors;
    }
}

Trade-off: Inheritance creates tight coupling between parent and child. Prefer composition over inheritance when the relationship is behavioral, not structural. If you are inheriting just to reuse methods, use composition instead.

Polymorphism

Compile-time (Static): Method overloading - same method name, different parameter signatures, resolved at compile time.

public class Calculator {
    public int add(int a, int b) { return a + b; }
    public double add(double a, double b) { return a + b; }
}

Runtime (Dynamic): Method overriding - subclass provides its own implementation of a parent method, resolved at runtime.

public abstract class Notification {
    public abstract void send(String message);
}

public class EmailNotification extends Notification {
    @Override
    public void send(String message) {
        System.out.println("Sending email: " + message);
    }
}

public class SMSNotification extends Notification {
    @Override
    public void send(String message) {
        System.out.println("Sending SMS: " + message);
    }
}

// Runtime dispatch
Notification notif = new EmailNotification();
notif.send("Your order is confirmed"); // Calls EmailNotification's implementation

Interview insight: When an interviewer asks you to add a new notification type, the right answer is to create a new subclass, not to add an if-else block. That is polymorphism at work.

3. SOLID Principles

S - Single Responsibility Principle (SRP)

A class should have one and only one reason to change.

// BAD - This class does too much
public class UserService {
    public void createUser(User user) { /* ... */ }
    public void sendWelcomeEmail(User user) { /* ... */ }
    public void generateUserReport(User user) { /* ... */ }
}

// GOOD - Separate concerns
public class UserService {
    public void createUser(User user) { /* ... */ }
}

public class EmailService {
    public void sendWelcomeEmail(User user) { /* ... */ }
}

public class ReportService {
    public void generateUserReport(User user) { /* ... */ }
}

Why it matters: When email logic changes, you do not want to risk breaking user creation logic. They are separate concerns and belong in separate classes.

O - Open/Closed Principle (OCP)

Classes should be open for extension, closed for modification.

// BAD - Adding a new discount type means modifying this class
public class DiscountCalculator {
    public double calculate(String discountType, double price) {
        if (discountType.equals("SEASONAL")) return price * 0.9;
        if (discountType.equals("EMPLOYEE")) return price * 0.7;
        return price; // Adding new type = modifying this method
    }
}

// GOOD
public interface DiscountStrategy {
    double apply(double price);
}

public class SeasonalDiscount implements DiscountStrategy {
    public double apply(double price) { return price * 0.9; }
}

public class EmployeeDiscount implements DiscountStrategy {
    public double apply(double price) { return price * 0.7; }
}

public class DiscountCalculator {
    public double calculate(DiscountStrategy strategy, double price) {
        return strategy.apply(price);
    }
}

Adding a new discount type now means creating a new class, not touching existing code.

L - Liskov Substitution Principle (LSP)

If S is a subtype of T, objects of type T should be replaceable with objects of type S without breaking the program.

// BAD - Violates LSP
public class Rectangle {
    protected int width, height;
    public void setWidth(int w) { this.width = w; }
    public void setHeight(int h) { this.height = h; }
    public int area() { return width * height; }
}

public class Square extends Rectangle {
    @Override
    public void setWidth(int w) { this.width = w; this.height = w; } // Breaks contract
    @Override
    public void setHeight(int h) { this.width = h; this.height = h; }
}
// Code that sets width=5, height=10 and expects area=50 breaks when given a Square

// GOOD - Use a common interface instead
public interface Shape {
    int area();
}

public class Rectangle implements Shape {
    private int width, height;
    public Rectangle(int width, int height) { this.width = width; this.height = height; }
    public int area() { return width * height; }
}

public class Square implements Shape {
    private int side;
    public Square(int side) { this.side = side; }
    public int area() { return side * side; }
}

Key test: If you have to override a method and throw an exception or change behavior in a way that breaks caller assumptions, you are violating LSP.

I - Interface Segregation Principle (ISP)

Clients should not be forced to depend on interfaces they do not use.

// BAD - Forced to implement irrelevant methods
public interface Worker {
    void work();
    void eat();
    void sleep();
}

public class RobotWorker implements Worker {
    public void work() { /* ... */ }
    public void eat() { throw new UnsupportedOperationException(); } // Robots don't eat
    public void sleep() { throw new UnsupportedOperationException(); }
}

// GOOD - Split into focused interfaces
public interface Workable {
    void work();
}

public interface HumanNeeds {
    void eat();
    void sleep();
}

public class HumanWorker implements Workable, HumanNeeds {
    public void work() { /* ... */ }
    public void eat() { /* ... */ }
    public void sleep() { /* ... */ }
}

public class RobotWorker implements Workable {
    public void work() { /* ... */ }
}

D - Dependency Inversion Principle (DIP)

High-level modules should not depend on low-level modules. Both should depend on abstractions.

// BAD - High-level class depends on a concrete low-level class
public class OrderService {
    private MySQLOrderRepository repository = new MySQLOrderRepository(); // Hardcoded dependency

    public void placeOrder(Order order) {
        repository.save(order);
    }
}

// GOOD - Depend on abstraction, inject the implementation
public interface OrderRepository {
    void save(Order order);
}

public class OrderService {
    private final OrderRepository repository;

    public OrderService(OrderRepository repository) { // Injected from outside
        this.repository = repository;
    }

    public void placeOrder(Order order) {
        repository.save(order);
    }
}

Now you can swap MySQLOrderRepository with MongoOrderRepository or a mock in tests without touching OrderService.

4. Design Patterns

Design patterns are reusable solutions to common design problems. They are not copy-paste templates. They are thinking tools.

Creational Patterns

Singleton

Ensures only one instance of a class exists throughout the application lifecycle.

When to use: Shared resources like configuration, connection pools, logging instances.

When NOT to use: If you find yourself using Singleton everywhere, you are probably hiding poor dependency management.

public class ConfigManager {
    private static volatile ConfigManager instance;
    private final Map<String, String> configs = new HashMap<>();

    private ConfigManager() {
        // Load configs from file or env
    }

    public static ConfigManager getInstance() {
        if (instance == null) {
            synchronized (ConfigManager.class) {
                if (instance == null) {
                    instance = new ConfigManager();
                }
            }
        }
        return instance;
    }

    public String get(String key) {
        return configs.getOrDefault(key, "");
    }
}

Common mistake: Not using volatile and double-checked locking. Without volatile, the JVM can reorder instructions and return a partially constructed object.

Trade-off: Makes unit testing hard because you cannot easily inject a mock. Prefer dependency injection over Singleton wherever possible.

Factory Method

Delegates the creation of objects to subclasses or dedicated factory methods. Decouples object creation from usage.

When to use: When the exact type of object to create depends on runtime conditions.

public interface Notification {
    void send(String message);
}

public class EmailNotification implements Notification {
    public void send(String message) { System.out.println("Email: " + message); }
}

public class PushNotification implements Notification {
    public void send(String message) { System.out.println("Push: " + message); }
}

public class NotificationFactory {
    public static Notification create(String type) {
        return switch (type) {
            case "EMAIL" -> new EmailNotification();
            case "PUSH" -> new PushNotification();
            default -> throw new IllegalArgumentException("Unknown notification type: " + type);
        };
    }
}

// Usage
Notification notif = NotificationFactory.create("EMAIL");
notif.send("Your order shipped");

Trade-off: Adding a new type means modifying the factory. Use an abstract factory or registry pattern if you have many types or need plugin-style extensibility.

Builder

Constructs a complex object step by step. Useful when an object has many optional fields and telescoping constructors become unreadable.

public class UserProfile {
    private final String userId;
    private final String name;
    private final String email;
    private final String phoneNumber;
    private final String address;

    private UserProfile(Builder builder) {
        this.userId = builder.userId;
        this.name = builder.name;
        this.email = builder.email;
        this.phoneNumber = builder.phoneNumber;
        this.address = builder.address;
    }

    public static class Builder {
        private final String userId; // Required
        private final String name;   // Required
        private String email;
        private String phoneNumber;
        private String address;

        public Builder(String userId, String name) {
            this.userId = userId;
            this.name = name;
        }

        public Builder email(String email) { this.email = email; return this; }
        public Builder phoneNumber(String phone) { this.phoneNumber = phone; return this; }
        public Builder address(String address) { this.address = address; return this; }

        public UserProfile build() {
            return new UserProfile(this);
        }
    }
}

// Usage - clean, readable, no positional argument confusion
UserProfile profile = new UserProfile.Builder("U001", "Alice")
    .email("alice@example.com")
    .address("Pune, India")
    .build();

Trade-off: More boilerplate. In production, Lombok's @Builder handles this, but knowing the manual pattern is essential for interviews.

Structural Patterns

Adapter

Wraps an incompatible interface to make it compatible with what the client expects.

Real-world analogy: A power plug adapter. The socket interface does not change, your device does not change, but the adapter bridges them.

// Third-party payment library we cannot modify
public class LegacyPaymentSDK {
    public String makeTransaction(String accountNo, float rupees) {
        return "SUCCESS";
    }
}

// Interface our system expects
public interface PaymentProcessor {
    boolean processPayment(String accountId, double amount);
}

// Adapter bridges the gap
public class PaymentAdapter implements PaymentProcessor {
    private final LegacyPaymentSDK legacySDK;

    public PaymentAdapter(LegacyPaymentSDK sdk) {
        this.legacySDK = sdk;
    }

    @Override
    public boolean processPayment(String accountId, double amount) {
        String result = legacySDK.makeTransaction(accountId, (float) amount);
        return "SUCCESS".equals(result);
    }
}

Decorator

Adds behavior to an object dynamically without modifying its class or creating subclasses for every combination.

When to use: When you need to stack behaviors, like adding logging, caching, retry logic to a service.

public interface DataReader {
    String read();
}

public class FileDataReader implements DataReader {
    public String read() { return "raw file data"; }
}

public class EncryptedDataReader implements DataReader {
    private final DataReader wrapped;
    public EncryptedDataReader(DataReader reader) { this.wrapped = reader; }
    public String read() { return decrypt(wrapped.read()); }
    private String decrypt(String data) { return "decrypted: " + data; }
}

public class CachedDataReader implements DataReader {
    private final DataReader wrapped;
    private String cache;
    public CachedDataReader(DataReader reader) { this.wrapped = reader; }
    public String read() {
        if (cache == null) cache = wrapped.read();
        return cache;
    }
}

// Usage - stacking decorators
DataReader reader = new CachedDataReader(new EncryptedDataReader(new FileDataReader()));
reader.read();

Trade-off: Debugging a deeply nested decorator chain can be confusing. Use clear naming and keep the chain shallow.

Behavioral Patterns

Strategy

Defines a family of algorithms, encapsulates each one, and makes them interchangeable at runtime.

When to use: When you have multiple ways to do something and you want to switch between them without conditionals.

public interface SortStrategy {
    void sort(int[] data);
}

public class QuickSort implements SortStrategy {
    public void sort(int[] data) { /* quicksort impl */ }
}

public class MergeSort implements SortStrategy {
    public void sort(int[] data) { /* mergesort impl */ }
}

public class DataProcessor {
    private SortStrategy strategy;

    public DataProcessor(SortStrategy strategy) {
        this.strategy = strategy;
    }

    public void setStrategy(SortStrategy strategy) {
        this.strategy = strategy;
    }

    public void process(int[] data) {
        strategy.sort(data);
    }
}

Interview insight: Every time you see a chain of if-else or switch based on a "type," ask yourself if Strategy pattern cleans it up.

Observer

Defines a one-to-many dependency so that when one object changes state, all its dependents are notified automatically.

When to use: Event systems, notification pipelines, pub-sub mechanisms.

public interface EventListener {
    void onEvent(String eventType, Object data);
}

public class EventManager {
    private final Map<String, List<EventListener>> listeners = new HashMap<>();

    public void subscribe(String eventType, EventListener listener) {
        listeners.computeIfAbsent(eventType, k -> new ArrayList<>()).add(listener);
    }

    public void publish(String eventType, Object data) {
        List<EventListener> eventListeners = listeners.getOrDefault(eventType, Collections.emptyList());
        for (EventListener listener : eventListeners) {
            listener.onEvent(eventType, data);
        }
    }
}

public class OrderService {
    private final EventManager eventManager;

    public OrderService(EventManager eventManager) {
        this.eventManager = eventManager;
    }

    public void placeOrder(Order order) {
        // save order
        eventManager.publish("ORDER_PLACED", order);
    }
}

public class EmailService implements EventListener {
    public void onEvent(String eventType, Object data) {
        System.out.println("Sending confirmation email for: " + eventType);
    }
}

Trade-off: If observers are slow or throw exceptions, they can affect the publisher. Consider async notification for production use.

Command

Encapsulates a request as an object, allowing you to parameterize, queue, log, or undo operations.

When to use: Undo/redo functionality, job queues, audit trails.

public interface Command {
    void execute();
    void undo();
}

public class TransferMoneyCommand implements Command {
    private final BankAccount from;
    private final BankAccount to;
    private final double amount;

    public TransferMoneyCommand(BankAccount from, BankAccount to, double amount) {
        this.from = from;
        this.to = to;
        this.amount = amount;
    }

    public void execute() {
        from.debit(amount);
        to.credit(amount);
    }

    public void undo() {
        to.debit(amount);
        from.credit(amount);
    }
}

public class CommandHistory {
    private final Deque<Command> history = new ArrayDeque<>();

    public void executeCommand(Command cmd) {
        cmd.execute();
        history.push(cmd);
    }

    public void undoLast() {
        if (!history.isEmpty()) history.pop().undo();
    }
}

5. Class Design and Relationships

Types of Relationships

// Association - loose relationship, no ownership
public class Driver {
    private Car car; // Driver can exist without car and vice versa
}

// Aggregation - weak ownership
public class Department {
    private List<Employee> employees; // Employees can exist even if department is dissolved
}

// Composition - strong ownership, child cannot exist without parent
public class House {
    private final List<Room> rooms; // Rooms are created and destroyed with the house

    public House(int numberOfRooms) {
        this.rooms = new ArrayList<>();
        for (int i = 0; i < numberOfRooms; i++) {
            rooms.add(new Room());
        }
    }
}

// Dependency - method-level, temporary use
public class InvoiceService {
    public void generate(PdfGenerator generator, Invoice invoice) { // Uses generator but doesn't own it
        generator.create(invoice);
    }
}

UML Class Diagram Basics

A class box has three sections:

+------------------+
|    ClassName     |
+------------------+
| - privateField   |
| + publicField    |
+------------------+
| + method(): void |
+------------------+

• - = private, + = public, # = protected
• Arrow with hollow triangle = Inheritance
• Dashed arrow = Dependency/Implementation
• Solid arrow with hollow head = inheritance
• Dashed arrow = dependency or implements
• Filled diamond = composition
• Hollow diamond = aggregation

When drawing in an interview, label your arrows. A diagram without labels communicates nothing.

6. Clean Code Practices

DRY, KISS, YAGNI

Naming Conventions

Good names are the cheapest form of documentation.

Layered Architecture

Controller Layer    → Handles HTTP, input parsing, response formatting
     |
Service Layer       → Business logic, orchestration, transaction management
     |
Repository Layer    → Data access, persistence, query execution

// Controller - handles request/response only
@RestController
public class OrderController {
    private final OrderService orderService;

    public OrderController(OrderService orderService) {
        this.orderService = orderService;
    }

    @PostMapping("/orders")
    public ResponseEntity<OrderResponse> createOrder(@Valid @RequestBody CreateOrderRequest request) {
        OrderDTO order = orderService.createOrder(request);
        return ResponseEntity.ok(new OrderResponse(order));
    }
}

// Service - owns the business logic
@Service
public class OrderService {
    private final OrderRepository orderRepository;
    private final InventoryService inventoryService;

    public OrderDTO createOrder(CreateOrderRequest request) {
        inventoryService.reserve(request.getItems());
        Order order = Order.from(request);
        Order saved = orderRepository.save(order);
        return OrderDTO.from(saved);
    }
}

// Repository - data access only
@Repository
public interface OrderRepository extends JpaRepository<Order, String> {
    List<Order> findByCustomerId(String customerId);
}

Common mistake: Putting business logic in controllers or database queries in service classes. These layers exist for a reason.

7. Common LLD Problems

Parking Lot

Key requirement: Track available spots, support multiple vehicle types, calculate fare.

Key classes:

public enum VehicleType { BIKE, CAR, TRUCK }

public class Vehicle {
    private final String licensePlate;
    private final VehicleType type;
}

public class ParkingSpot {
    private final String spotId;
    private final VehicleType allowedType;
    private boolean isOccupied;
    private Vehicle parkedVehicle;
}

public class ParkingFloor {
    private final String floorId;
    private final List<ParkingSpot> spots;

    public Optional<ParkingSpot> findAvailableSpot(VehicleType type) {
        return spots.stream()
            .filter(s -> s.getAllowedType() == type && !s.isOccupied())
            .findFirst();
    }
}

public class ParkingLot {
    private static ParkingLot instance; // Singleton - one lot per JVM
    private final List<ParkingFloor> floors;
    private final FareCalculator fareCalculator;

    public Ticket park(Vehicle vehicle) {
        for (ParkingFloor floor : floors) {
            Optional<ParkingSpot> spot = floor.findAvailableSpot(vehicle.getType());
            if (spot.isPresent()) {
                spot.get().park(vehicle);
                return new Ticket(vehicle, spot.get(), LocalDateTime.now());
            }
        }
        throw new ParkingLotFullException("No available spot for: " + vehicle.getType());
    }
}

public interface FareCalculator {
    double calculate(Ticket ticket);
}

public class HourlyFareCalculator implements FareCalculator {
    private final Map<VehicleType, Double> ratePerHour;

    public double calculate(Ticket ticket) {
        long hours = ChronoUnit.HOURS.between(ticket.getEntryTime(), LocalDateTime.now());
        return ratePerHour.get(ticket.getVehicle().getType()) * Math.max(1, hours);
    }
}

Design decisions:

• ParkingLot is Singleton because there is only one physical lot
• FareCalculator is an interface (Strategy pattern) so pricing logic can change without touching ParkingLot
• findAvailableSpot returns Optional to force callers to handle the "no spot" case explicitly

Library Management System

Key classes: Book, BookCopy, Member, Loan, LibraryCatalog, LoanService

public class Book {
    private final String isbn;
    private final String title;
    private final String author;
}

public class BookCopy {
    private final String copyId;
    private final Book book;
    private boolean isAvailable;
}

public class Member {
    private final String memberId;
    private final List<Loan> activeLoans;

    public boolean canBorrow() {
        return activeLoans.size() < MAX_LOANS_ALLOWED;
    }
}

public class Loan {
    private final BookCopy copy;
    private final Member member;
    private final LocalDate borrowDate;
    private LocalDate returnDate;
    private LoanStatus status;
}

public class LoanService {
    private final LoanRepository loanRepository;
    private final NotificationService notificationService;

    public Loan issueBook(Member member, BookCopy copy) {
        if (!member.canBorrow()) throw new LoanLimitExceededException(member.getMemberId());
        if (!copy.isAvailable()) throw new BookNotAvailableException(copy.getCopyId());

        copy.setAvailable(false);
        Loan loan = new Loan(copy, member, LocalDate.now());
        loanRepository.save(loan);
        notificationService.sendBorrowConfirmation(member, copy.getBook());
        return loan;
    }
}

Design decisions:

• Separate Book (logical entity) from BookCopy (physical instance). A library has one "Clean Code" book but may have 5 physical copies.
• canBorrow() is a domain method on Member, not a service-level check. Domain logic belongs in domain objects.

Elevator System

Key classes: Elevator, ElevatorController, Request, ElevatorScheduler

public enum Direction { UP, DOWN, IDLE }

public class Elevator {
    private final int elevatorId;
    private int currentFloor;
    private Direction direction;
    private ElevatorStatus status;
    private final TreeSet<Integer> floorsToVisit = new TreeSet<>();

    public void addFloorRequest(int floor) {
        floorsToVisit.add(floor);
    }

    public void move() {
        if (floorsToVisit.isEmpty()) { direction = Direction.IDLE; return; }
        int nextFloor = direction == Direction.UP
            ? floorsToVisit.first()
            : floorsToVisit.last();
        currentFloor = nextFloor;
        floorsToVisit.remove(nextFloor);
    }
}

public interface ElevatorScheduler {
    Elevator selectElevator(List<Elevator> elevators, int requestedFloor, Direction direction);
}

public class NearestElevatorScheduler implements ElevatorScheduler {
    public Elevator selectElevator(List<Elevator> elevators, int floor, Direction direction) {
        return elevators.stream()
            .min(Comparator.comparingInt(e -> Math.abs(e.getCurrentFloor() - floor)))
            .orElseThrow();
    }
}

Design decisions:

• ElevatorScheduler is an interface. You can swap in SCAN algorithm, LOOK algorithm, or nearest-first without changing elevator internals.
• TreeSet for floor requests because it keeps floors sorted, making directional traversal efficient.

Splitwise

Key classes: User, Expense, Split, Balance, ExpenseService

public abstract class Split {
    protected final User user;
    protected double amount;
    public abstract void calculateAmount(double totalAmount, int numberOfPeople);
}

public class EqualSplit extends Split {
    public EqualSplit(User user) { super(user); }
    public void calculateAmount(double total, int people) { this.amount = total / people; }
}

public class PercentSplit extends Split {
    private final double percent;
    public PercentSplit(User user, double percent) { super(user); this.percent = percent; }
    public void calculateAmount(double total, int people) { this.amount = total * percent / 100; }
}

public class Expense {
    private final String expenseId;
    private final User paidBy;
    private final double totalAmount;
    private final List<Split> splits;
    private final String description;
}

public class BalanceService {
    // Returns simplified balances: who owes whom and how much
    public Map<User, Map<User, Double>> calculateBalances(List<Expense> expenses) {
        Map<User, Map<User, Double>> balances = new HashMap<>();
        for (Expense expense : expenses) {
            User payer = expense.getPaidBy();
            for (Split split : expense.getSplits()) {
                if (!split.getUser().equals(payer)) {
                    balances.computeIfAbsent(split.getUser(), k -> new HashMap<>())
                        .merge(payer, split.getAmount(), Double::sum);
                }
            }
        }
        return balances;
    }
}

Design decisions:

• Split is abstract. Different split types (equal, percentage, exact) are subclasses. Adding a new split type does not touch existing code.

Shopping Cart

Key classes: Product, CartItem, Cart, PricingEngine, OrderService

public class Product {
    private final String productId;
    private final String name;
    private final double basePrice;
    private final int stockCount;
}

public class CartItem {
    private final Product product;
    private int quantity;

    public double getSubtotal() {
        return product.getBasePrice() * quantity;
    }
}

public class Cart {
    private final String cartId;
    private final String userId;
    private final Map<String, CartItem> items = new LinkedHashMap<>();

    public void addItem(Product product, int quantity) {
        items.merge(product.getProductId(),
            new CartItem(product, quantity),
            (existing, newItem) -> { existing.increaseQuantity(quantity); return existing; });
    }

    public double getTotal() {
        return items.values().stream().mapToDouble(CartItem::getSubtotal).sum();
    }
}

public interface PricingRule {
    double apply(Cart cart);
}

public class CouponDiscountRule implements PricingRule {
    private final String couponCode;
    private final double discountPercent;

    public double apply(Cart cart) {
        return cart.getTotal() * (1 - discountPercent / 100);
    }
}

8. Error Handling and Logging

Exception Handling Best Practices

• Catch specific exceptions, not Exception or Throwable
• Never silently swallow exceptions
• Clean up resources in finally or use try-with-resources
• Fail fast: validate inputs at system boundaries

// BAD
try {
    processOrder(order);
} catch (Exception e) {
    // silently ignored - debugging nightmare
}

// GOOD
try {
    processOrder(order);
} catch (InsufficientInventoryException e) {
    log.warn("Inventory check failed for order {}: {}", order.getId(), e.getMessage());
    throw new OrderCreationException("Product out of stock", e);
} catch (PaymentFailedException e) {
    log.error("Payment failed for order {}", order.getId(), e);
    notificationService.alertUser(order.getUserId(), "Payment failed");
    throw e;
}

Custom Exceptions

Design your exception hierarchy to mirror your domain.

// Base exception for the domain
public class ApplicationException extends RuntimeException {
    private final String errorCode;

    public ApplicationException(String message, String errorCode) {
        super(message);
        this.errorCode = errorCode;
    }

    public ApplicationException(String message, String errorCode, Throwable cause) {
        super(message, cause);
        this.errorCode = errorCode;
    }

    public String getErrorCode() { return errorCode; }
}

public class ResourceNotFoundException extends ApplicationException {
    public ResourceNotFoundException(String resource, String id) {
        super(resource + " not found with id: " + id, "RESOURCE_NOT_FOUND");
    }
}

public class ValidationException extends ApplicationException {
    private final List<String> violations;

    public ValidationException(List<String> violations) {
        super("Validation failed", "VALIDATION_ERROR");
        this.violations = violations;
    }
}

Logging Levels

// Structured logging - always include context
log.info("Order created successfully | orderId={} userId={} amount={}",
    order.getId(), order.getUserId(), order.getTotal());

log.error("Payment failed | orderId={} gateway={} errorCode={}",
    order.getId(), gateway.getName(), response.getErrorCode());

// NEVER log sensitive data
// BAD: log.info("User logged in | password={}", password);
// GOOD: log.info("User logged in | userId={}", userId);

9. Concurrency Basics

Threads vs Processes

Race Condition

When two threads access shared mutable state concurrently and the outcome depends on the timing of execution.

// BAD - race condition on counter
public class Counter {
    private int count = 0;

    public void increment() {
        count++; // Read-increment-write is NOT atomic
    }
}

// GOOD - use AtomicInteger for single variable
public class Counter {
    private final AtomicInteger count = new AtomicInteger(0);

    public void increment() {
        count.incrementAndGet(); // Atomic operation
    }
}

Synchronization

Use synchronized for compound operations or multi-variable state that must be consistent.

public class BankAccount {
    private double balance;
    private final Object lock = new Object();

    public void transfer(BankAccount target, double amount) {
        // Always lock in consistent order to prevent deadlock
        BankAccount first = this.hashCode() < target.hashCode() ? this : target;
        BankAccount second = first == this ? target : this;

        synchronized (first.lock) {
            synchronized (second.lock) {
                if (this.balance < amount) throw new InsufficientFundsException();
                this.balance -= amount;
                target.balance += amount;
            }
        }
    }
}

Deadlock

Occurs when two or more threads are each waiting for a lock held by the other, resulting in a permanent standstill.

Four conditions required (remove any one to prevent deadlock):

1. Mutual exclusion
2. Hold and wait
3. No preemption
4. Circular wait

Prevention: Always acquire locks in a consistent global order. The bank transfer example above uses hash code ordering to ensure lock A is always acquired before lock B.

Thread-Safe Design

// Prefer immutable objects - they are inherently thread-safe
public final class Money {
    private final double amount;
    private final String currency;

    public Money(double amount, String currency) {
        this.amount = amount;
        this.currency = currency;
    }

    public Money add(Money other) {
        if (!this.currency.equals(other.currency)) throw new CurrencyMismatchException();
        return new Money(this.amount + other.amount, this.currency); // Returns new object
    }
}

// Use concurrent collections
private final Map<String, Session> sessions = new ConcurrentHashMap<>();
private final BlockingQueue<Task> taskQueue = new LinkedBlockingQueue<>();

// Use ExecutorService instead of raw threads
ExecutorService executor = Executors.newFixedThreadPool(10);
executor.submit(() -> processOrder(order));

Modern practices: Prefer java.util.concurrent over raw synchronized. Use CompletableFuture for async pipelines. Keep shared state minimal. Prefer message passing over shared mutable state.

10. API and Service Layer Design

DTO vs Entity

// Entity - internal, mapped to DB table
@Entity
@Table(name = "users")
public class User {
    @Id
    private String userId;
    private String name;
    private String email;
    private String passwordHash; // Should NEVER appear in API response
    private LocalDateTime createdAt;
}

// DTO - external, safe to return from API
public class UserResponseDTO {
    private String userId;
    private String name;
    private String email;
    // No passwordHash - intentional

    public static UserResponseDTO from(User user) {
        UserResponseDTO dto = new UserResponseDTO();
        dto.userId = user.getUserId();
        dto.name = user.getName();
        dto.email = user.getEmail();
        return dto;
    }
}

// Request DTO - what the API accepts
public class CreateUserRequest {
    @NotBlank(message = "Name is required")
    private String name;

    @Email(message = "Invalid email format")
    @NotBlank
    private String email;

    @Size(min = 8, message = "Password must be at least 8 characters")
    private String password;
}

Validation

Validate at the boundary - controllers are the entry point. Do not let invalid data leak into service or domain layers.

@RestController
@RequestMapping("/api/users")
public class UserController {
    private final UserService userService;

    @PostMapping
    public ResponseEntity<UserResponseDTO> createUser(
            @Valid @RequestBody CreateUserRequest request) { // @Valid triggers Bean Validation
        UserResponseDTO user = userService.createUser(request);
        return ResponseEntity.status(HttpStatus.CREATED).body(user);
    }
}

// Global exception handler
@RestControllerAdvice
public class GlobalExceptionHandler {
    @ExceptionHandler(MethodArgumentNotValidException.class)
    public ResponseEntity<ErrorResponse> handleValidationErrors(MethodArgumentNotValidException ex) {
        List<String> errors = ex.getBindingResult().getFieldErrors().stream()
            .map(e -> e.getField() + ": " + e.getDefaultMessage())
            .collect(Collectors.toList());
        return ResponseEntity.badRequest().body(new ErrorResponse("VALIDATION_ERROR", errors));
    }

    @ExceptionHandler(ResourceNotFoundException.class)
    public ResponseEntity<ErrorResponse> handleNotFound(ResourceNotFoundException ex) {
        return ResponseEntity.status(HttpStatus.NOT_FOUND)
            .body(new ErrorResponse(ex.getErrorCode(), ex.getMessage()));
    }
}

Dependency Injection

Do not create dependencies inside classes. Inject them from outside.

// Constructor injection - preferred over field injection (@Autowired on field)
@Service
public class OrderService {
    private final OrderRepository orderRepository;
    private final InventoryService inventoryService;
    private final NotificationService notificationService;

    // Spring injects these at startup
    public OrderService(OrderRepository orderRepository,
                        InventoryService inventoryService,
                        NotificationService notificationService) {
        this.orderRepository = orderRepository;
        this.inventoryService = inventoryService;
        this.notificationService = notificationService;
    }
}

Why constructor injection over field injection: Makes dependencies explicit, enables immutability with final, and makes the class testable without a Spring container.

11. Testing Basics

Unit Testing

Tests a single unit of logic in isolation. Dependencies are mocked.

@ExtendWith(MockitoExtension.class)
class OrderServiceTest {

    @Mock
    private OrderRepository orderRepository;

    @Mock
    private InventoryService inventoryService;

    @InjectMocks
    private OrderService orderService;

    @Test
    void placeOrder_shouldSaveOrder_whenInventoryAvailable() {
        // Arrange
        Order order = new Order("ORD001", "USER001", List.of(new OrderItem("PROD1", 2)));
        when(inventoryService.isAvailable("PROD1", 2)).thenReturn(true);
        when(orderRepository.save(any(Order.class))).thenReturn(order);

        // Act
        Order result = orderService.placeOrder(order);

        // Assert
        assertNotNull(result);
        assertEquals("ORD001", result.getOrderId());
        verify(inventoryService).reserve("PROD1", 2);
        verify(orderRepository).save(order);
    }

    @Test
    void placeOrder_shouldThrow_whenInventoryInsufficient() {
        Order order = new Order("ORD001", "USER001", List.of(new OrderItem("PROD1", 100)));
        when(inventoryService.isAvailable("PROD1", 100)).thenReturn(false);

        assertThrows(InsufficientInventoryException.class, () -> orderService.placeOrder(order));
        verify(orderRepository, never()).save(any()); // Order should not be saved
    }
}

Writing Testable Code

Test Structure: Arrange-Act-Assert

Always structure tests in three clear sections:

@Test
void calculateDiscount_shouldApply10Percent_forSeasonalCoupon() {
    // Arrange - set up the state
    Cart cart = new Cart("CART001", "USER001");
    cart.addItem(new Product("PROD1", "Laptop", 50000.0), 1);
    DiscountStrategy seasonal = new SeasonalDiscount(10);

    // Act - execute the behavior being tested
    double finalPrice = seasonal.apply(cart.getTotal());

    // Assert - verify the outcome
    assertEquals(45000.0, finalPrice, 0.001);
}

Common mistake: Testing implementation details instead of behavior. Your test should not care whether calculateDiscount used a loop or a stream internally. It should only verify the output.

Quick Reference Summary

Conclusion

Low Level Design is not about memorizing patterns or writing complex code. It is about building systems that are easy to understand, extend, and maintain over time.

If you can apply OOP principles, follow SOLID, choose the right design patterns, and think in terms of trade-offs, you are already ahead of most developers in both interviews and real-world engineering.

Use this cheatsheet as a reference, but focus on practicing real problems and writing clean designs consistently.

If you found this helpful, consider sharing it with your friends and peers who are preparing for backend and system design interviews.

Designing scalable systems can feel overwhelming with so many moving parts—caching, load balancing, databases, microservices. This cheatsheet breaks down High Level Design (HLD) into a simple, structured format you can actually understand.

Whether you're preparing for interviews or building real systems, it covers the key concepts in one place—focusing on how things work, the trade-offs involved, and how to make practical design decisions.

1. System Design Fundamentals

What is High Level Design (HLD)?

High Level Design is the bird's-eye view of a system. You're not writing code here — you're deciding what components exist, how they talk to each other, and how the system behaves under load or failure.

Think of it as the blueprint before construction. HLD answers: What are we building and how does it fit together?

HLD vs LLD

Key System Design Goals

• Scalability – Can the system handle 10x traffic without a rewrite?
• Reliability – Does it work correctly even when parts fail?
• Availability – Is it up when users need it? (Measured as uptime %)
• Consistency – Do all nodes agree on the same data at the same time?

These four goals often conflict. More on that in the CAP Theorem section.

Functional vs Non-Functional Requirements

Always split requirements before designing anything.

Functional requirements — what the system does

• User can upload a photo
• System sends an OTP on login
• Search returns results in under 2 seconds

Non-functional requirements — how well it does it

• System must support 1 million concurrent users
• 99.99% uptime SLA
• Data must not be lost during a crash

In interviews, listing NFRs early signals maturity. It tells the interviewer you're thinking about the hard stuff.

Latency vs Throughput

• Latency — time for one request to complete (e.g., 50ms per API call)
• Throughput — number of requests processed per second (e.g., 10,000 RPS)

You often trade one for the other. Batching improves throughput but increases latency per individual item. Caching reduces latency but adds complexity.

Rule of thumb: optimize latency for user-facing requests, throughput for background pipelines.

2. Core Building Blocks

Client, Server, API

• Client — the consumer of data. Browser, mobile app, another service.
• Server — processes requests, applies business logic, returns responses.
• API — the contract between client and server. Defines what's available and how to call it.

Client  --[HTTP Request]-->  API Server  --[Query]-->  Database
Client  <--[JSON Response]--  API Server  <--[Result]--  Database

Load Balancer

Sits between clients and servers. Distributes incoming traffic so no single server gets overwhelmed.

• Prevents single points of failure
• Enables horizontal scaling
• Handles health checks (removes dead servers from rotation)

Example: Nginx, AWS ALB, HAProxy

Database — SQL vs NoSQL

Cache

A fast, in-memory store that keeps frequently accessed data close to the application layer, reducing database load.

• Redis — in-memory key-value store. Supports strings, hashes, sorted sets, pub/sub. Used for sessions, leaderboards, rate limiting.
• CDN (Content Delivery Network) — caches static assets (images, JS, CSS) on edge servers geographically close to users. Think Cloudflare, Akamai.

Message Queue

Decouples producers and consumers. Producer puts a message on the queue; consumer picks it up independently.

• Kafka — high-throughput, distributed log. Great for event streaming, audit logs.
• RabbitMQ — traditional message broker. Better for task queues, simpler routing.

Use when: you don't need a synchronous response, you're processing in bulk, or you want to absorb traffic spikes.

3. Scalability Concepts

Vertical vs Horizontal Scaling

Stateless vs Stateful Services

• Stateless — each request carries all needed context. Server doesn't remember previous requests. Easy to scale horizontally. REST APIs are stateless by design.
• Stateful — server remembers user state (e.g., WebSocket connections, game sessions). Harder to scale, requires sticky sessions or shared state store.

Design services to be stateless wherever possible. Move state to a shared store like Redis.

Auto Scaling

Automatically adds or removes instances based on metrics like CPU, memory, or request queue depth.

• Scale out — add instances when load increases
• Scale in — remove instances when load drops

Cloud providers (AWS Auto Scaling, GCP Managed Instance Groups) handle this natively. Set minimum, maximum, and target thresholds.

CAP Theorem

A distributed system can guarantee only two of three properties at the same time:

• Consistency — every read gets the most recent write
• Availability — every request gets a (non-error) response
• Partition Tolerance — system keeps working even if nodes can't communicate

In practice, network partitions happen. So you're choosing between CP or AP systems.

Consistency Models

• Strong consistency — after a write, every subsequent read sees that write. Safer but slower.
• Eventual consistency — writes propagate eventually; reads may be stale for a short window. Faster but requires careful design.

Real-world example: your Twitter follower count may be slightly off for a few seconds after someone unfollows. That's eventual consistency in action — and totally acceptable there.

4. Database Design

SQL vs NoSQL Decision Making

Ask these questions:

• Do you need complex joins or transactions? → SQL
• Is your schema changing frequently? → NoSQL
• Do you need horizontal scale from day one? → NoSQL
• Is data relational (users, orders, products)? → SQL
• Are you storing logs, events, or documents? → NoSQL

There's no universal winner. Most mature systems use both.

Indexing Basics

An index is a data structure that speeds up reads at the cost of slower writes and extra storage.

• Without index: full table scan, O(n)
• With index: B-tree or hash lookup, O(log n)

-- Creates an index on email column for fast lookups
CREATE INDEX idx_users_email ON users(email);

Index columns used in WHERE, JOIN, and ORDER BY clauses. Avoid over-indexing — every index slows down INSERT/UPDATE.

Sharding

Splitting a large database into smaller pieces (shards) across multiple servers. Each shard holds a subset of data.

• Range-based sharding — shard by ID range (1–1M on shard 1, 1M–2M on shard 2)
• Hash-based sharding — shard by hash of a key (user_id % N)
• Directory-based sharding — a lookup table maps each key to a shard

Challenges: joins across shards are expensive, resharding is painful, hotspots can occur.

Replication

Copying data from one database node (primary) to one or more others (replicas).

• Synchronous replication — primary waits for replica to confirm before responding. Stronger consistency, higher latency.
• Asynchronous replication — primary doesn't wait. Faster, but replica may lag.

Used for fault tolerance, data redundancy, and geographic distribution.

Read Replicas

A common pattern: write to the primary, read from replicas.

Write Request  -->  Primary DB
Read Request   -->  Replica 1 / Replica 2 / Replica 3

This offloads read traffic from the primary. Works well when your read:write ratio is high (most web apps are read-heavy).

5. Caching Strategies

Why Caching?

Databases are slow compared to memory. A query that takes 50ms from disk takes under 1ms from Redis. Caching reduces latency, saves DB resources, and improves throughput.

Use caching for: frequently read, rarely written data — user profiles, product listings, config values.

Cache-Aside (Lazy Loading)

The most common pattern. Application checks cache first; if miss, fetches from DB and populates cache.

1. App checks cache for key
2. If HIT → return cached value
3. If MISS → query DB
4. Store result in cache with TTL
5. Return result to caller

Good for read-heavy workloads. Cache only stores what's actually requested.

Write-Through

Every write goes to cache and DB simultaneously.

1. App writes data
2. Cache is updated
3. DB is updated
4. Acknowledge success

Keeps cache always fresh. Downside: writes are slightly slower.

Write-Back (Write-Behind)

App writes to cache only. Cache asynchronously flushes to DB later.

1. App writes to cache
2. Acknowledge success (fast)
3. Cache flushes to DB asynchronously

Very fast writes. Risk: if cache crashes before flush, data is lost. Use only when you can tolerate brief inconsistency.

Cache Invalidation

The hardest part. How do you know when to expire a cached value?

• TTL (Time To Live) — set an expiry time. Simple and effective for most cases.
• Event-driven invalidation — when underlying data changes, explicitly delete or update the cache key.
• Write-through — keeps cache current by design.

"There are only two hard things in computer science: cache invalidation and naming things." — Phil Karlton

6. Load Balancing

L4 vs L7 Load Balancing

Use L7 when you need smart routing (e.g., /api/* to one server pool, /static/* to another).

Load Balancing Algorithms

• Round Robin — requests go to servers in rotation. Simple, works when servers are equal.
• Weighted Round Robin — servers with more capacity get proportionally more requests.
• Least Connections — next request goes to the server with fewest active connections. Good for long-lived connections.
• IP Hash / Sticky Sessions — same client always goes to same server. Useful for stateful apps.
• Random — randomly pick a server. Surprisingly effective at scale.

Health Checks

Load balancers regularly ping servers to check if they're alive.

• Active health checks — LB sends HTTP request to /health every N seconds
• Passive health checks — LB monitors real traffic; marks server unhealthy after N consecutive failures

A dead server gets removed from rotation automatically. When it recovers, it gets added back.

7. API Design Basics

REST Principles

REST (Representational State Transfer) is an architectural style, not a protocol. The six principles:

• Stateless — no client session stored on server
• Client-Server — clear separation of concerns
• Uniform Interface — consistent, predictable URLs and methods
• Cacheable — responses can be cached
• Layered System — client doesn't know if it's hitting server directly or via proxy
• Code on Demand (optional) — server can send executable code

GET    /users/123       → fetch user
POST   /users           → create user
PUT    /users/123       → replace user
PATCH  /users/123       → partial update
DELETE /users/123       → delete user

Idempotency

An operation is idempotent if calling it multiple times produces the same result as calling it once.

• GET, PUT, DELETE — idempotent
• POST — not idempotent by default (creates a new resource each time)

Why it matters: networks are unreliable. Clients retry requests. If your order-creation endpoint isn't idempotent, a retry creates a duplicate order. Use an idempotency key (unique request ID in header) to prevent this.

Pagination

Never return all records in one response. Use one of:

Offset pagination can be slow at high offsets (LIMIT 20 OFFSET 100000 is expensive). Cursor-based is preferred at scale.

Rate Limiting

Protects your API from abuse and overload.

• Token bucket — refills tokens at a fixed rate. Requests consume tokens.
• Fixed window — count requests per time window (e.g., 100 per minute)
• Sliding window — smoother than fixed window, avoids edge spikes

Return 429 Too Many Requests with a Retry-After header.

API Versioning

Plan for breaking changes from day one.

• URL versioning — /api/v1/users (most common, easy to understand)
• Header versioning — Accept: application/vnd.api.v2+json
• Query param — /api/users?version=2

Never remove a version without a deprecation period. Give consumers time to migrate.

8. Messaging and Asynchronous Systems

Pub/Sub Model

Publishers emit events to a topic. Subscribers listen to topics they care about. Neither knows about the other.

OrderService (publisher)  →  "order.created" topic  →  EmailService (subscriber)
                                                    →  InventoryService (subscriber)
                                                    →  AnalyticsService (subscriber)

This is loosely coupled. Adding a new subscriber doesn't change the publisher at all.

Message Queues

Point-to-point. One producer puts a message on the queue; one consumer picks it up.

• Guarantees processing — message stays in queue until acknowledged
• Enables backpressure — if consumers are slow, queue buffers the load
• Good for task processing (image resizing, email sending, PDF generation)

Event-Driven Architecture

System components communicate entirely through events rather than direct calls.

Benefits:

• Services are decoupled
• Easy to add new functionality (just subscribe to existing events)
• Natural audit log (event stream = history of what happened)

Challenges:

• Harder to debug (no single call stack)
• Eventual consistency by nature
• Need to handle out-of-order events

Retry and Dead-Letter Queues

When message processing fails, you don't want to lose it.

• Retry — attempt reprocessing N times with exponential backoff
• Dead-Letter Queue (DLQ) — after max retries, move message here for investigation

Queue  →  Consumer fails
       →  Retry 1 (after 1s)
       →  Retry 2 (after 2s)
       →  Retry 3 (after 4s)
       →  Move to DLQ → alert on-call engineer

Always set up a DLQ in production. Silently dropping failed messages is dangerous.

9. Microservices Architecture

Monolith vs Microservices

Start with a monolith. Break it apart when you feel the pain — not before.

Service Communication

Synchronous (Request/Response)

• REST over HTTP, gRPC
• Caller waits for a response
• Simpler to reason about
• Creates tight coupling; if downstream is slow, caller is slow

Asynchronous (Event/Message)

• Kafka, RabbitMQ, SQS
• Caller fires and forgets
• Better resilience and decoupling
• Harder to debug, eventual consistency

Use sync for user-facing flows where you need an immediate answer. Use async for background processing.

API Gateway

A single entry point for all client requests. Routes to the appropriate downstream service.

Responsibilities:

• Request routing
• Authentication and authorization
• Rate limiting
• SSL termination
• Request/response transformation
• Logging

Mobile App  →  API Gateway  →  UserService
Web App     →               →  ProductService
                            →  OrderService

Examples: AWS API Gateway, Kong, Nginx

Service Discovery

Services need to know each other's addresses. In dynamic environments (containers, auto-scaling), IPs change constantly.

• Client-side discovery — client queries a service registry (e.g., Consul, Eureka) and picks an instance
• Server-side discovery — load balancer handles it; client just calls the LB

Kubernetes handles this natively via DNS-based service discovery.

10. Reliability and Fault Tolerance

Redundancy

Having more than one instance of a component so that if one fails, others take over.

• Active-active: multiple instances all handle traffic
• Active-passive: one primary, one standby that activates on failure

Apply redundancy to: servers, databases, load balancers, entire data centers.

Failover

The automatic (or manual) process of switching to a redundant component when the primary fails.

• Automatic failover — system detects failure, switches within seconds (e.g., database primary/replica promotion)
• Manual failover — requires human intervention, acceptable for planned maintenance

RTO (Recovery Time Objective) defines how quickly you need to recover. Failover speed must beat your RTO.

Circuit Breaker

Prevents cascading failures. When a downstream service is failing, stop sending it requests for a period.

Three states:

• Closed — everything normal, requests pass through
• Open — too many failures, requests immediately rejected with an error (fast fail)
• Half-Open — after a timeout, allow a test request through; if it succeeds, close the circuit

if circuit.state == OPEN:
    return fallback_response()  # don't even try

try:
    response = call_downstream_service()
    circuit.record_success()
except Exception:
    circuit.record_failure()
    if circuit.failure_rate > threshold:
        circuit.open()

Hystrix (Java), Resilience4j, and Polly (.NET) are popular implementations.

Retries and Timeouts

• Timeout — don't wait forever for a response. Set a reasonable limit (e.g., 500ms for user-facing, 5s for background).
• Retry — on transient failures, try again. Use exponential backoff to avoid hammering a struggling service.

Attempt 1: fail → wait 1s
Attempt 2: fail → wait 2s
Attempt 3: fail → wait 4s
Attempt 4: give up, return error

Always combine retries with jitter (randomized delay) to avoid thundering herd.

11. Security Basics

Authentication vs Authorization

These are not the same thing.

• Authentication — who are you? (Login, verify identity)
• Authorization — what are you allowed to do? (Permissions, roles)

Auth flow:

User logs in → system verifies identity (AuthN) → issues token
User requests /admin → system checks if user has admin role (AuthZ) → allow or deny

Common patterns: JWT tokens, OAuth 2.0, session cookies, API keys.

HTTPS and Encryption

• HTTPS — HTTP over TLS. Encrypts data in transit. Non-negotiable for production. Get certs from Let's Encrypt (free).
• Encryption at rest — encrypt sensitive database fields (PII, passwords). Never store passwords in plaintext — use bcrypt or Argon2 hashing.
• Encryption in transit — TLS between all internal services, not just public-facing ones.

API Security

• Validate all inputs — never trust client data. Prevent SQL injection, XSS.
• Use HTTPS everywhere
• Rotate API keys regularly
• Scope tokens minimally — don't give a read-only client write permissions
• Validate JWTs properly — check signature, expiry, and audience
• Use CORS headers correctly — don't set Access-Control-Allow-Origin: * in production

Rate Limiting and Throttling

Both protect your system from abuse, but have a subtle difference:

• Rate limiting — hard cap on requests per time window (block after N requests)
• Throttling — slow down responses rather than blocking (return 429 or add delay)

Apply rate limiting per IP, per user, and per API key. Different limits for different tiers (free vs paid users).

12. Monitoring and Observability

The goal of observability is to understand what your system is doing — especially when things go wrong. The three pillars are logs, metrics, and traces.

Logging

A log is a timestamped record of an event.

Best practices:

• Use structured logging (JSON format) — easier to parse and query
• Include correlation IDs to trace a request across services
• Log at appropriate levels: DEBUG, INFO, WARN, ERROR
• Don't log sensitive data (passwords, PII, tokens)
• Centralize logs (ELK Stack, Datadog, CloudWatch)

{
  "timestamp": "2025-03-01T10:22:00Z",
  "level": "ERROR",
  "service": "order-service",
  "correlation_id": "abc-123",
  "message": "Payment gateway timeout",
  "user_id": "user-456"
}

Metrics

Numerical measurements over time. Used for dashboards and alerting.

Four golden signals (from Google SRE):

• Latency — how long requests take
• Traffic — how many requests per second
• Errors — rate of failed requests
• Saturation — how "full" your system is (CPU, memory, queue depth)

Tools: Prometheus + Grafana, Datadog, AWS CloudWatch.

Distributed Tracing

A trace follows a single request across multiple services, showing where time is spent.

Request enters API Gateway (0ms)
  → UserService (5ms)
  → ProductService (20ms)
	  → DB query (18ms)  ← bottleneck
  → Response (27ms total)

Tools: Jaeger, Zipkin, AWS X-Ray, Datadog APM. Every service adds a span to the trace with timing and metadata.

Alerts

Alerts notify on-call engineers when something needs attention.

Good alerting principles:

• Alert on symptoms, not causes (alert on high error rate, not "CPU is high")
• Set thresholds carefully — too sensitive creates alert fatigue
• Every alert should be actionable — if you can't do anything about it, don't alert on it
• Use PagerDuty, OpsGenie, or similar for on-call routing
• Write runbooks: for each alert, document what it means and what to do

Quick Reference: Key Concepts at a Glance

Conclusion

High Level Design is not about memorizing patterns — it’s about understanding trade-offs, scalability, and real-world constraints. Every system you design will require balancing consistency, availability, performance, and cost based on the problem you're solving.

This cheatsheet gives you a solid foundation, but true mastery comes from thinking in systems, practicing real-world problems, and continuously learning from existing architectures.

Keep building, keep breaking things, and keep improving

If you found this helpful, feel free to share it with your friends and help them level up in System Design too

Hibernate is an open-source Object Relational Mapping (ORM) framework for Java that simplifies database interactions by allowing developers to work with Java objects instead of writing raw SQL queries. It handles mapping between object-oriented code and relational databases, manages connections, caching, and transactions, and is widely used in modern backend applications.

1. Hibernate Fundamentals

What is Hibernate

Hibernate is an open-source ORM (Object Relational Mapping) framework for Java. It simplifies database interaction by letting you work with Java objects instead of raw SQL. It implements the JPA specification and sits between your application and the database.

• Developed by Gavin King, now maintained by Red Hat
• Implements JPA (Jakarta Persistence API)
• Handles SQL generation, connection pooling, caching, and transactions

ORM - Object Relational Mapping

ORM bridges the gap between object-oriented Java code and relational databases. Instead of writing INSERT or SELECT statements, you work with plain Java objects (POJOs).

Why Hibernate over JDBC

JDBC works, but it is verbose, error-prone, and forces you to manage everything manually. Hibernate abstracts all of that away.

Hibernate Architecture Overview

Application
    |
    v
SessionFactory  <--- Configuration (hibernate.cfg.xml or annotations)
    |
    v
Session  <--- Wraps JDBC Connection
    |
    v
Transaction
    |
    v
Database (via JDBC Driver)

Key components:

• Configuration - loads Hibernate settings
• SessionFactory - heavyweight, one per database, thread-safe
• Session - lightweight, one per request/transaction, not thread-safe
• Transaction - wraps DB operations
• Query / Criteria - for executing HQL or JPQL queries

2. Core Concepts

Entity Class and POJO

A POJO (Plain Old Java Object) becomes a Hibernate entity when you annotate it with @Entity. It must have a no-arg constructor and a primary key field.

@Entity
@Table(name = "employees")
public class Employee {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    private String name;
    private String department;

    // no-arg constructor required
    public Employee() {}

    // getters and setters
}

Rules for an entity class:

• Must not be final
• Must have a public or protected no-arg constructor
• Must have at least one @Id field
• Fields should be private with getters/setters

Session and SessionFactory

SessionFactory is created once when the application starts. It is expensive to build but cheap to use. Think of it as a factory that produces Session objects.

Session is the primary interface to interact with the database. It represents a single unit of work and wraps a JDBC connection.

// Build SessionFactory (do this once, on startup)
SessionFactory sessionFactory = new Configuration()
    .configure("hibernate.cfg.xml")
    .buildSessionFactory();

// Open a session for each unit of work
Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();

// ... do work ...

tx.commit();
session.close();

Persistence Context

The persistence context is an in-memory cache of entities that Hibernate manages during a session. Every entity loaded or saved goes into this context.

• Tracks changes automatically (dirty checking)
• Ensures identity - same DB row returns the same Java object
• Synchronizes state to DB at flush time

Three entity states:

• Transient - object created but not associated with any session
• Persistent - associated with an active session, tracked by Hibernate
• Detached - was persistent, session is now closed

Employee emp = new Employee(); // Transient

session.save(emp);             // Persistent - Hibernate tracks this

session.close();               // emp becomes Detached

First-Level Cache

First-level cache is scoped to the Session. It is enabled by default and cannot be disabled. When you load the same entity twice in the same session, the second call hits cache, not the database.

Employee e1 = session.get(Employee.class, 1L); // hits DB
Employee e2 = session.get(Employee.class, 1L); // hits L1 cache, no DB call

System.out.println(e1 == e2); // true - same object reference

3. Configuration

hibernate.cfg.xml

This is the traditional XML-based configuration file placed in src/main/resources.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE hibernate-configuration PUBLIC
    "-//Hibernate/Hibernate Configuration DTD 3.0//EN"
    "<http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd>">

<hibernate-configuration>
    <session-factory>

        <!-- Database connection -->
        <property name="hibernate.connection.driver_class">com.mysql.cj.jdbc.Driver</property>
        <property name="hibernate.connection.url">jdbc:mysql://localhost:3306/mydb</property>
        <property name="hibernate.connection.username">root</property>
        <property name="hibernate.connection.password">password</property>

        <!-- Dialect -->
        <property name="hibernate.dialect">org.hibernate.dialect.MySQL8Dialect</property>

        <!-- Show SQL in console -->
        <property name="hibernate.show_sql">true</property>
        <property name="hibernate.format_sql">true</property>

        <!-- Schema generation -->
        <property name="hibernate.hbm2ddl.auto">update</property>

        <!-- Entity class mapping -->
        <mapping class="com.example.Employee"/>

    </session-factory>
</hibernate-configuration>

Annotation-Based Configuration

In modern Spring Boot projects, application.properties handles Hibernate config:

spring.datasource.url=jdbc:mysql://localhost:3306/mydb
spring.datasource.username=root
spring.datasource.password=password
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver

spring.jpa.hibernate.ddl-auto=update
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.format_sql=true
spring.jpa.database-platform=org.hibernate.dialect.MySQL8Dialect

Key Properties Explained

hbm2ddl.auto values:

• create - drops and recreates schema on startup
• create-drop - drops schema on shutdown (good for tests)
• update - updates schema without dropping data
• validate - validates schema, throws error if mismatch
• none - does nothing (recommended for production)

4. Mapping Annotations

Basic Annotations

@Entity                         // marks this class as a Hibernate entity
@Table(name = "employees")      // maps to a specific table name
public class Employee {

    @Id                                                    // primary key
    @GeneratedValue(strategy = GenerationType.IDENTITY)    // auto-increment
    @Column(name = "emp_id")                               // maps to specific column
    private Long id;

    @Column(name = "full_name", nullable = false, length = 100)
    private String name;

    @Column(unique = true)
    private String email;

    @Transient   // this field will NOT be persisted
    private String tempToken;
}

@GeneratedValue strategies:

Relationship Annotations

@OneToOne

One entity maps to exactly one instance of another.

@Entity
public class Employee {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @OneToOne(cascade = CascadeType.ALL)
    @JoinColumn(name = "address_id", referencedColumnName = "id")
    private Address address;
}

@OneToMany and @ManyToOne

One department has many employees. Each employee belongs to one department.

// Department side (One)
@Entity
public class Department {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @OneToMany(mappedBy = "department", cascade = CascadeType.ALL, fetch = FetchType.LAZY)
    private List<Employee> employees = new ArrayList<>();
}

// Employee side (Many)
@Entity
public class Employee {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @ManyToOne(fetch = FetchType.LAZY)
    @JoinColumn(name = "department_id")
    private Department department;
}

• mappedBy tells Hibernate that the other side owns the relationship
• The owning side (the one with @JoinColumn) controls the foreign key

@ManyToMany

Students can enroll in many courses. Each course has many students.

@Entity
public class Student {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @ManyToMany
    @JoinTable(
        name = "student_course",
        joinColumns = @JoinColumn(name = "student_id"),
        inverseJoinColumns = @JoinColumn(name = "course_id")
    )
    private List<Course> courses = new ArrayList<>();
}

@Entity
public class Course {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @ManyToMany(mappedBy = "courses")
    private List<Student> students = new ArrayList<>();
}

CascadeType Quick Reference

5. CRUD Operations

Save and Persist

Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();

Employee emp = new Employee();
emp.setName("Alice");
emp.setDepartment("Engineering");

session.save(emp);      // Hibernate-specific, returns generated ID
// OR
session.persist(emp);   // JPA standard, returns void

tx.commit();
session.close();

• save() returns the generated identifier immediately
• persist() is JPA-standard and does not guarantee immediate INSERT

Get and Load

// get() - hits DB immediately, returns null if not found
Employee emp = session.get(Employee.class, 1L);

// load() - returns a proxy, hits DB only when you access fields
// throws ObjectNotFoundException if not found (not null)
Employee emp = session.load(Employee.class, 1L);

Use get() when you are unsure whether the record exists. Use load() only when you are certain the record is there.

Update

// For detached objects
session.update(emp);

// merge() - safer, works for both detached and transient
session.merge(emp);

// Automatic dirty checking - no explicit update needed for persistent objects
Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();

Employee emp = session.get(Employee.class, 1L); // persistent
emp.setName("Bob");  // just change the field

tx.commit(); // Hibernate detects the change and fires UPDATE automatically
session.close();

Delete

Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();

Employee emp = session.get(Employee.class, 1L);
session.delete(emp); // or session.remove(emp) in JPA

tx.commit();
session.close();

6. HQL and JPQL

Basics of HQL

HQL (Hibernate Query Language) is object-oriented SQL. You write queries against entity class names and field names, not table names and column names. Hibernate translates them to native SQL.

// Basic SELECT
List<Employee> employees = session
    .createQuery("FROM Employee", Employee.class)
    .getResultList();

// WHERE clause
List<Employee> result = session
    .createQuery("FROM Employee WHERE department = :dept", Employee.class)
    .setParameter("dept", "Engineering")
    .getResultList();

// SELECT specific fields
List<String> names = session
    .createQuery("SELECT e.name FROM Employee e", String.class)
    .getResultList();

// JOIN
List<Employee> result = session
    .createQuery("FROM Employee e JOIN FETCH e.department WHERE e.department.name = :deptName", Employee.class)
    .setParameter("deptName", "Engineering")
    .getResultList();

Named Queries

Named queries are defined once (usually on the entity) and reused by name. They are parsed and validated at startup, which catches errors early.

@Entity
@NamedQuery(
    name = "Employee.findByDepartment",
    query = "FROM Employee WHERE department.name = :deptName"
)
public class Employee {
    // ...
}

// Usage
List<Employee> employees = session
    .createNamedQuery("Employee.findByDepartment", Employee.class)
    .setParameter("deptName", "Engineering")
    .getResultList();

Pagination

List<Employee> page = session
    .createQuery("FROM Employee ORDER BY name", Employee.class)
    .setFirstResult(0)   // offset - starts from 0
    .setMaxResults(10)   // page size
    .getResultList();

7. Fetching Strategies

Lazy vs Eager Loading

Fetching strategy decides when related entities are loaded from the database.

// Lazy - employees not loaded until you call dept.getEmployees()
@OneToMany(mappedBy = "department", fetch = FetchType.LAZY)
private List<Employee> employees;

// Eager - employees always loaded with department
@OneToMany(mappedBy = "department", fetch = FetchType.EAGER)
private List<Employee> employees;

General rule: prefer LAZY loading. Use EAGER only when you always need the related data.

N+1 Problem

The N+1 problem happens when loading N parent entities triggers N additional queries to load their children, resulting in N+1 total queries to the database.

// BAD - triggers N+1
List<Department> departments = session
    .createQuery("FROM Department", Department.class)
    .getResultList();

for (Department dept : departments) {
    dept.getEmployees(); // fires a separate query for EACH department
}
// Result: 1 query for departments + N queries for employees = N+1 queries

Fix: use JOIN FETCH

// GOOD - loads everything in one query
List<Department> departments = session
    .createQuery("FROM Department d JOIN FETCH d.employees", Department.class)
    .getResultList();

8. Caching

First-Level Cache

• Enabled by default, always active
• Scoped to the Session - lives and dies with the session
• Cannot be turned off
• Automatically used when you call session.get() multiple times for the same entity

Second-Level Cache

• Shared across sessions within the same SessionFactory
• Disabled by default - you must configure it
• Useful for read-heavy, rarely-changing data (like reference tables)
• Common providers: Ehcache, Infinispan, Hazelcast

<!-- hibernate.cfg.xml -->
<property name="hibernate.cache.use_second_level_cache">true</property>
<property name="hibernate.cache.region.factory_class">
    org.hibernate.cache.ehcache.EhCacheRegionFactory
</property>

@Entity
@Cache(usage = CacheConcurrencyStrategy.READ_WRITE)  // enable L2 cache for this entity
public class Country {
    @Id
    private Long id;
    private String name;
}

Cache Concurrency Strategies:

Query Cache - caches query result sets (not just entities). Must be explicitly enabled per query.

session.createQuery("FROM Country", Country.class)
    .setCacheable(true)
    .getResultList();

9. Transactions

Transaction Lifecycle

Every database write operation must happen inside a transaction. Hibernate does not auto-commit by default.

Session session = sessionFactory.openSession();
Transaction tx = null;

try {
    tx = session.beginTransaction();

    // your database operations here
    session.save(employee);

    tx.commit();         // writes changes to DB

} catch (Exception e) {
    if (tx != null) tx.rollback();  // undo everything on error
    e.printStackTrace();
} finally {
    session.close();     // always close the session
}

With Spring's @Transactional:

@Service
public class EmployeeService {

    @Transactional
    public void saveEmployee(Employee emp) {
        employeeRepository.save(emp);
        // Spring manages begin, commit, and rollback automatically
    }
}

ACID Properties

Isolation Levels:

10. Schema Design and Best Practices

Normalization vs Denormalization

• Normalization - reduce redundancy, split data into related tables. Better for write-heavy systems. Easier to maintain.
• Denormalization - combine related data into fewer tables. Better for read-heavy systems. Improves query performance at the cost of redundancy.

Hibernate works better with normalized schemas but can handle denormalized designs too.

Efficient Mapping Strategies

• Use @ManyToOne instead of @OneToMany as the owning side where possible
• Avoid bidirectional relationships unless you genuinely need to navigate both ways
• Use mappedBy correctly - only one side should own the relationship
• Prefer List over Set for ordered collections, but Set for unordered to avoid duplicates

// Good practice - explicit column definitions
@Column(name = "created_at", nullable = false, updatable = false)
private LocalDateTime createdAt;

// Good practice - use @Index for frequently queried columns
@Table(name = "employees", indexes = {
    @Index(name = "idx_email", columnList = "email"),
    @Index(name = "idx_department", columnList = "department_id")
})

Performance Considerations

• Always close sessions - use try-with-resources or finally blocks
• Use pagination for large result sets - never load all rows
• Avoid FetchType.EAGER on collections
• Use JOIN FETCH only when you need related data in the same query
• Enable second-level cache for reference/static data
• Use batch inserts for bulk operations

// Batch inserts - configure batch size in properties
// hibernate.jdbc.batch_size=50

for (int i = 0; i < 1000; i++) {
    session.save(new Employee("emp_" + i));
    if (i % 50 == 0) {
        session.flush();  // write batch to DB
        session.clear();  // clear L1 cache to avoid memory issues
    }
}

11. Common Pitfalls

LazyInitializationException

This is one of the most common Hibernate errors. It happens when you try to access a lazily loaded collection or association after the session is already closed.

// BAD - session closes after this method returns
public Department findDepartment(Long id) {
    Session session = sessionFactory.openSession();
    Department dept = session.get(Department.class, id);
    session.close(); // session closed here
    return dept;
}

// Later...
dept.getEmployees().size(); // LazyInitializationException - session is gone

Fixes:

• Use JOIN FETCH to load required associations eagerly within the session
• Use @Transactional to keep session open until the method returns
• Use DTOs to avoid returning entities outside the session boundary
• Use Hibernate.initialize(dept.getEmployees()) explicitly before closing session

N+1 Query Issue

Already covered in Section 7. Quick summary:

• Root cause: lazy loading inside a loop
• Detection: enable show_sql and count your queries
• Fix: use JOIN FETCH or @EntityGraph

// @EntityGraph approach (JPA standard)
@EntityGraph(attributePaths = {"employees"})
List<Department> findAll(); // Spring Data JPA method

Improper Session Handling

• Never share a Session across threads - it is not thread-safe
• Never leave sessions open - causes connection pool exhaustion
• Always use transactions for writes - reads may work without them but writes are unreliable without explicit transaction boundaries

// BAD - storing session as a field
public class EmployeeService {
    private Session session = sessionFactory.openSession(); // wrong
}

// GOOD - open and close per operation
public Employee findById(Long id) {
    try (Session session = sessionFactory.openSession()) {
        return session.get(Employee.class, id);
    }
}

12. Hibernate vs JPA

The Difference

JPA (Jakarta Persistence API) is a specification - it defines a standard set of interfaces and annotations for ORM in Java. Hibernate is the most popular implementation of that specification.

Think of JPA as the interface and Hibernate as the class that implements it.

Code Comparison

// JPA style (portable, works with EclipseLink too)
EntityManagerFactory emf = Persistence.createEntityManagerFactory("my-pu");
EntityManager em = emf.createEntityManager();
em.getTransaction().begin();
em.persist(employee);
em.getTransaction().commit();
em.close();

// Hibernate native style
SessionFactory sf = new Configuration().configure().buildSessionFactory();
Session session = sf.openSession();
session.beginTransaction();
session.save(employee);
session.getTransaction().commit();
session.close();

When to Use What

• Use JPA when you want portability and do not need Hibernate-specific features. Standard choice for enterprise apps.
• Use Hibernate-specific APIs when you need features like Hibernate filters, second-level cache fine-tuning, batch processing helpers, or advanced fetch profiles.
• In practice: use JPA annotations and EntityManager for all standard operations, and reach for Hibernate-specific APIs only when JPA falls short.

Quick Reference Card

Session Methods

Common Annotations Summary

hbm2ddl.auto Quick Reference

Conclusion

Hibernate simplifies database interaction by abstracting complex SQL into clean, object-oriented code, allowing developers to focus on business logic instead of low-level data handling. With features like automatic mapping, caching, and transaction management, and its seamless use with frameworks like Spring Boot, Hibernate remains an essential tool for building scalable and maintainable Java backend applications. If you found this helpful, consider sharing it with your friends and peers.

MongoDB is a powerful NoSQL database built for modern applications that demand flexibility, speed, and scalability. Instead of rigid tables and schemas, it stores data in JSON-like documents, making it easy to model real-world data and iterate quickly as requirements evolve.

Designed for high performance and distributed systems, MongoDB enables developers to handle large-scale data with ease through features like indexing, aggregation, replication, and sharding. It is widely used in building scalable backends, real-time applications, and microservices architectures.

1. MongoDB Fundamentals

What is MongoDB?

MongoDB is a NoSQL, document-oriented database that stores data as flexible, JSON-like documents instead of rows and columns. It is schema-less by default, meaning each document in a collection can have different fields.

Why MongoDB over SQL?

• Stores hierarchical/nested data naturally
• Horizontally scalable
• Great for unstructured or semi-structured data
• Fast reads on large datasets with proper indexing

JSON vs BSON

MongoDB stores data as BSON internally but exposes it as JSON to the developer.

Database → Collection → Document Hierarchy

MongoDB Instance
  └── Database (e.g., "shopDB")
        └── Collection (e.g., "products")
              └── Document (e.g., { name: "Laptop", price: 999 })

• Database - container for collections
• Collection - equivalent to a table in SQL
• Document - equivalent to a row; a BSON object

The _id Field

Every document must have a unique _id field. If you don't provide one, MongoDB auto-generates an ObjectId.

{
  _id: ObjectId("64a3f8c2d1e2a3b4c5d6e7f8"),
  name: "Alice",
  age: 28
}

• ObjectId is 12 bytes: 4-byte timestamp + 5-byte random + 3-byte incrementing counter
• You can use any unique value as _id (string, integer, etc.)

Installation Checklist

• MongoDB Community Server - the core database engine
• MongoDB Compass - GUI for exploring and querying data visually
• mongosh - the official MongoDB shell for terminal-based interaction

# Start mongosh
mongosh

# Select or create a database
use shopDB

# Check current db
db

# List all databases
show dbs

# List collections
show collections

2. Core CRUD Operations

Insert

// Insert a single document
db.users.insertOne({
  name: "Ravi",
  email: "ravi@example.com",
  age: 25,
  city: "Mumbai"
})

// Insert multiple documents
db.users.insertMany([
  { name: "Priya", age: 22, city: "Delhi" },
  { name: "Arjun", age: 30, city: "Bangalore" }
])

• insertOne() returns insertedId
• insertMany() returns an array of insertedIds

Read

// Find all documents
db.users.find()

// Find with a filter
db.users.find({ city: "Mumbai" })

// Find one document
db.users.findOne({ name: "Ravi" })

// Pretty print in older shell
db.users.find().pretty()

Update

// Update a single document
db.users.updateOne(
  { name: "Ravi" },
  { $set: { age: 26 } }
)

// Update multiple documents
db.users.updateMany(
  { city: "Mumbai" },
  { $set: { country: "India" } }
)

// Increment a value
db.users.updateOne(
  { name: "Ravi" },
  { $inc: { age: 1 } }
)

// Push to an array
db.users.updateOne(
  { name: "Ravi" },
  { $push: { hobbies: "photography" } }
)

// Pull from an array
db.users.updateOne(
  { name: "Ravi" },
  { $pull: { hobbies: "photography" } }
)

Delete

// Delete one
db.users.deleteOne({ name: "Ravi" })

// Delete many
db.users.deleteMany({ city: "Delhi" })

// Delete all documents in a collection
db.users.deleteMany({})

Common Update Operators

3. Advanced Querying Techniques

MongoDB provides a flexible query language to filter, combine conditions, and shape results. Queries operate on documents and can leverage indexes for efficient execution.

Comparison Operators

Comparison operators filter documents based on field values, similar to SQL WHERE conditions.

// Greater than
db.products.find({ price: { $gt: 500 } })

// Less than or equal
db.products.find({ price: { $lte: 1000 } })

// Equal (explicit)
db.products.find({ status: { $eq: "active" } })

// Not equal
db.products.find({ status: { $ne: "inactive" } })

// In a list
db.products.find({ category: { $in: ["Electronics", "Books"] } })

// Not in a list
db.products.find({ category: { $nin: ["Clothing"] } })

These operators are evaluated per document. MongoDB can efficiently use indexes with operators like $eq, $gt, and $lt. The $in operator is useful for matching multiple values but can become slower with very large arrays.

Logical Operators

Logical operators combine multiple conditions to form complex queries.

// $and - both conditions must match
db.users.find({
  $and: [{ age: { $gte: 18 } }, { city: "Mumbai" }]
})

// $or - either condition matches
db.users.find({
  $or: [{ city: "Delhi" }, { city: "Mumbai" }]
})

// $not - negates a condition
db.users.find({
  age: { $not: { $gt: 30 } }
})

// $nor - none of the conditions match
db.users.find({
  $nor: [{ city: "Delhi" }, { age: { $lt: 18 } }]
})

MongoDB implicitly applies $and when multiple conditions are specified in a single query object. $or queries may require proper indexing for good performance. $not and $nor are generally less efficient and should be used carefully on large datasets.

Projection

Projection controls which fields are returned in the result set.

// Return only name and city, exclude _id
db.users.find({}, { name: 1, city: 1, _id: 0 })

// Exclude a field
db.users.find({}, { password: 0 })

Projection reduces the amount of data transferred from the database and can improve performance. It also enables covered queries when all required fields are present in an index. Inclusion and exclusion cannot be mixed in the same projection, except for _id.

Sorting

Sorting arranges documents based on specified fields.

// Ascending (1), Descending (-1)
db.products.find().sort({ price: 1 })     // cheapest first
db.products.find().sort({ price: -1 })    // most expensive first
db.products.find().sort({ name: 1, price: -1 }) // multi-sort

Sorting without an index may result in in-memory operations, which are slower. For better performance, indexes should be created on fields used in sorting. Compound indexes should match the sort order.

Limiting and Skipping

Used to control the number of documents returned and implement pagination.

// Return only 5 documents
db.products.find().limit(5)

// Skip first 10, then get 5 (pagination)
db.products.find().skip(10).limit(5)

The limit() function reduces the number of documents returned, improving efficiency. The skip() function can become inefficient on large datasets because MongoDB still scans skipped documents internally.

For scalable pagination, cursor-based approaches (e.g., using _id or indexed fields) are preferred over large skip() values.

4. Indexing and Query Optimization

Without an index, MongoDB performs a collection scan - checking every document. With an index, it jumps directly to matching documents. Think of it like a book index versus reading every page.

Creating Indexes

// Single field index
db.users.createIndex({ email: 1 })

// Compound index (multiple fields)
db.orders.createIndex({ userId: 1, createdAt: -1 })

// Unique index
db.users.createIndex({ email: 1 }, { unique: true })

// Text index (full-text search)
db.articles.createIndex({ content: "text" })

// List all indexes on a collection
db.users.getIndexes()

// Drop an index
db.users.dropIndex("email_1")

Text Search with Text Index

db.articles.find({ $text: { $search: "mongodb tutorial" } })

Explain - Understand Query Performance

db.users.find({ email: "ravi@example.com" }).explain("executionStats")

Key fields to check in explain output:

• COLLSCAN - no index used, bad for large collections
• IXSCAN - index used, efficient
• nReturned - documents returned
• totalDocsExamined - documents scanned

Indexing Best Practices

• Index fields used in find(), sort(), and $lookup
• Compound indexes follow the ESR rule: Equality first, Sort next, Range last
• Too many indexes slow down writes
• Use explain() to verify your index is being used
• TTL indexes can auto-expire documents (useful for sessions, logs)

// TTL index - auto delete after 3600 seconds
db.sessions.createIndex({ createdAt: 1 }, { expireAfterSeconds: 3600 })

5. Schema Design and Data Modeling

MongoDB is schema-flexible, but that doesn't mean you should design carelessly. The right schema design depends on your read/write patterns.

Embedding vs Referencing

Embedding - store related data inside the same document

// Embedded address inside user
{
  _id: ObjectId("..."),
  name: "Priya",
  address: {
    street: "123 MG Road",
    city: "Pune",
    pincode: "411001"
  }
}

Referencing - store the related document's _id as a foreign key

// User document
{ _id: ObjectId("u1"), name: "Priya" }

// Order document references user
{ _id: ObjectId("o1"), userId: ObjectId("u1"), total: 4500 }

When to Embed vs Reference

Denormalization

MongoDB often duplicates data intentionally (denormalization) to avoid expensive joins at read time. For example, storing a user's name alongside each order even though it also exists in the users collection - this way you never need a join to display order history.

Schema Design Principles

• Design your schema around how your application queries data, not how the data naturally relates
• Avoid unbounded arrays inside documents (keep arrays small and finite)
• A document size limit of 16 MB applies in MongoDB
• Use ObjectId references when relationships are complex or data is large

6. Aggregation Framework Deep Dive

The aggregation pipeline processes documents through a series of stages, where each stage transforms the data. Think of it as a conveyor belt - documents go in, get filtered, grouped, reshaped, and come out the other end.

Basic Pipeline Syntax

db.collection.aggregate([
  { stage1 },
  { stage2 },
  { stage3 }
])

$match - Filter Documents

Works like find(). Always put $match first to reduce documents early.

db.orders.aggregate([
  { $match: { status: "delivered" } }
])

$group - Group and Aggregate

// Total revenue per city
db.orders.aggregate([
  {
    $group: {
      _id: "$city",
      totalRevenue: { $sum: "$amount" },
      orderCount: { $sum: 1 },
      avgOrder: { $avg: "$amount" }
    }
  }
])

Common $group accumulators:

$project - Reshape Documents

db.users.aggregate([
  {
    $project: {
      fullName: { $concat: ["$firstName", " ", "$lastName"] },
      age: 1,
      _id: 0
    }
  }
])

$sort and $limit

db.orders.aggregate([
  { $match: { status: "delivered" } },
  { $group: { _id: "$customerId", total: { $sum: "$amount" } } },
  { $sort: { total: -1 } },
  { $limit: 10 }
])

This pipeline finds the top 10 customers by total spending.

$unwind - Flatten Arrays

// Each element of the array becomes its own document
db.orders.aggregate([
  { $unwind: "$items" }
])

$lookup - Join Collections

db.orders.aggregate([
  {
    $lookup: {
      from: "users",          // collection to join
      localField: "userId",   // field in orders
      foreignField: "_id",    // field in users
      as: "userDetails"       // output array field
    }
  }
])

7. Relationships and Joins in MongoDB

MongoDB is not a relational database, but you can model and query relationships effectively.

One-to-One

// Embedded (preferred for one-to-one)
{
  _id: ObjectId("u1"),
  name: "Arjun",
  profile: {
    bio: "Developer",
    website: "arjun.dev"
  }
}

One-to-Many

// Author has many books - reference approach
{ _id: ObjectId("a1"), name: "Chetan Bhagat" }

{ _id: ObjectId("b1"), title: "2 States", authorId: ObjectId("a1") }
{ _id: ObjectId("b2"), title: "3 Mistakes", authorId: ObjectId("a1") }

// Query using $lookup
db.books.aggregate([
  {
    $lookup: {
      from: "authors",
      localField: "authorId",
      foreignField: "_id",
      as: "author"
    }
  }
])

Many-to-Many

Use an intermediary collection (like a junction table in SQL) or embed an array of references.

// Students and courses - array of references
{ _id: ObjectId("s1"), name: "Neha", enrolledCourses: [ObjectId("c1"), ObjectId("c2")] }
{ _id: ObjectId("c1"), title: "MongoDB Basics" }

// Or a separate enrollment collection
{ studentId: ObjectId("s1"), courseId: ObjectId("c1"), enrolledAt: ISODate("2024-01-10") }

8. Transactions and Data Consistency

By default, individual document operations in MongoDB are atomic. If you need to update multiple documents atomically, you need multi-document transactions (available from MongoDB 4.0+ with replica sets).

ACID in MongoDB

Multi-Document Transactions

const session = client.startSession()

try {
  session.startTransaction()

  await db.collection("accounts").updateOne(
    { _id: "acc1" },
    { $inc: { balance: -500 } },
    { session }
  )

  await db.collection("accounts").updateOne(
    { _id: "acc2" },
    { $inc: { balance: 500 } },
    { session }
  )

  await session.commitTransaction()
} catch (err) {
  await session.abortTransaction()
} finally {
  session.endSession()
}

Key Points About Transactions

• Transactions require a replica set or sharded cluster - they don't work on standalone instances
• Keep transactions short and fast to avoid lock contention
• Transactions have a default timeout of 60 seconds
• Use transactions only when you truly need multi-document atomicity; single-document operations are cheaper

9. MongoDB Atlas (Cloud Database)

MongoDB Atlas is the fully managed cloud version of MongoDB. It handles provisioning, backups, scaling, and security for you.

Setting Up a Free Cluster

1. Go to cloud.mongodb.com and sign up
2. Create a new project
3. Build a cluster - choose M0 (free tier) for learning
4. Create a database user with username and password
5. Whitelist your IP address under Network Access
6. Click Connect to get your connection string

Connection String Format

mongodb+srv://<username>:<password>@cluster0.xxxxx.mongodb.net/<dbname>?retryWrites=true&w=majority

Atlas Features Worth Knowing

• Atlas Search - full-text search powered by Lucene
• Atlas Triggers - run serverless functions on database events
• Atlas Data API - access data over HTTP without a driver
• Backups - automated point-in-time backups
• Performance Advisor - suggests indexes based on slow queries

10. Security and Access Control

Authentication

MongoDB supports several authentication mechanisms:

• SCRAM (default) - username/password
• X.509 - certificate-based
• LDAP - enterprise only
• Kerberos - enterprise only

# Connect with authentication
mongosh "mongodb://username:password@localhost:27017/dbname"

Built-in Roles

Creating a User

use admin

db.createUser({
  user: "appUser",
  pwd: "securePassword123",
  roles: [
    { role: "readWrite", db: "shopDB" }
  ]
})

Data Protection Best Practices

• Always enable authentication (disabled by default in local dev)
• Use TLS/SSL for connections in production
• Enable encryption at rest (Atlas does this automatically)
• Never expose MongoDB port (27017) directly to the internet
• Rotate credentials regularly
• Use schema validation to prevent malformed data

// Adding schema validation to a collection
db.createCollection("users", {
  validator: {
    $jsonSchema: {
      bsonType: "object",
      required: ["name", "email"],
      properties: {
        email: { bsonType: "string" }
      }
    }
  }
})

11. Scaling and Performance Optimization

Replication

A replica set is a group of MongoDB instances that hold the same data. It provides high availability and data redundancy.

Primary Node──writes──>  Secondary Node 1
                 └────>  Secondary Node 2

• Reads go to the primary by default; can be configured to read from secondaries
• If the primary goes down, an automatic election picks a new primary
• Minimum recommended: 3 nodes (1 primary + 2 secondaries)
• Write concern and read preference control consistency vs performance tradeoffs

// Connect to replica set
mongosh "mongodb://host1:27017,host2:27017,host3:27017/dbname?replicaSet=myReplicaSet"

Sharding

Sharding distributes data horizontally across multiple machines. Use it when a single machine can no longer handle the data volume or throughput.

• Shard - a replica set holding a subset of data
• Mongos - the router that directs queries to the right shard
• Config Servers - store cluster metadata
• Shard Key - the field used to distribute data across shards

// Enable sharding on a database
sh.enableSharding("shopDB")

// Shard a collection by a key
sh.shardCollection("shopDB.orders", { customerId: "hashed" })

Choosing a good shard key:

• High cardinality (many unique values)
• Even distribution of reads and writes
• Avoid monotonically increasing fields like timestamps as the sole shard key (causes hotspots)

Query Optimization Checklist

• Always use explain("executionStats") to analyze slow queries
• Ensure commonly queried fields are indexed
• Avoid using $where (executes JavaScript - very slow)
• Use projections to return only needed fields
• Avoid large skip() values on big collections; use cursor-based pagination instead
• Keep documents lean - don't store fields you never query
• Use covered queries (index covers all fields in query + projection - no document fetch needed)

// Covered query example
// Index: { email: 1, name: 1 }
db.users.find(
  { email: "test@example.com" },
  { name: 1, _id: 0 }
)
// MongoDB can return the result purely from the index, without touching documents

Quick Reference Card

Most Used Commands

// Database
use dbName
show dbs
show collections
db.dropDatabase()

// CRUD
db.col.insertOne({})
db.col.insertMany([])
db.col.find({ field: value })
db.col.findOne({ field: value })
db.col.updateOne({ filter }, { $set: {} })
db.col.updateMany({ filter }, { $set: {} })
db.col.deleteOne({ filter })
db.col.deleteMany({ filter })

// Aggregation
db.col.aggregate([ {$match:{}}, {$group:{}}, {$sort:{}} ])

// Indexes
db.col.createIndex({ field: 1 })
db.col.getIndexes()
db.col.dropIndex("indexName")

// Explain
db.col.find({}).explain("executionStats")

Operator Quick Reference

Conclusion

MongoDB offers a flexible and powerful approach to data modeling, making it an excellent choice for modern, scalable applications. With its document-based structure, rich query capabilities, and support for indexing, aggregation, and distributed systems, it enables developers to build high-performance backends with ease.

Mastering MongoDB is not just about learning queries, but understanding how to design schemas, optimize performance, and scale systems effectively. By applying the concepts in this guide, you can confidently use MongoDB in real-world applications and production environments.

If you found this helpful, consider sharing it with your friends and peers who are learning or working with MongoDB.

Most developers interact with Spring Boot at the surface level: write a controller, add a service, annotate a method with @Transactional, and ship it. But understanding what happens inside the application from the moment a request arrives to the moment a response is sent back is what separates a developer who writes code from an engineer who understands systems.

This article walks through the complete internal lifecycle of a Spring Boot HTTP request. It covers how threads are assigned, how security context is established, how transactions bind to the request, how database connections are managed, and where your application breaks under load. Every section describes behavior that is actually happening inside your JVM right now — most of it invisible unless you know where to look.

Overview — The Complete Request Journey

When a client sends an HTTP request to a Spring Boot application, it enters a well-defined pipeline. Each stage has a specific role. Nothing happens randomly.

At the highest level, the journey looks like this:

1. The client sends a TCP connection and HTTP request to the embedded server (Tomcat by default).
2. The server accepts the connection and assigns a worker thread from its thread pool.
3. The request passes through the Servlet filter chain, which includes Spring Security filters.
4. The authenticated request reaches DispatcherServlet, which routes it to the correct controller method.
5. The controller delegates work to a service, which may open a database transaction and interact with a repository.
6. A database connection is borrowed from the connection pool, queries are executed, and the connection is returned.
7. The response is serialized (usually to JSON) and written back over the TCP connection.
8. The thread is released back to the pool, ready for the next request.

Every single one of these steps has performance implications, memory implications, and failure modes. Let us go through each one carefully.

Request Entry and Thread Assignment

What is a Thread

A thread is the smallest unit of execution in a Java process. When your Spring Boot application starts, it runs inside a single JVM process. That process can run many threads concurrently, each with its own execution stack, meaning each thread tracks its own method calls, local variables, and program counter independently.

Threads share heap memory, which is where objects live. But each thread has private stack memory, which is where its method call frames live. This distinction matters because it is exactly how Spring Boot isolates request state.

How the Server Assigns a Thread

Spring Boot ships with an embedded Tomcat server by default. When Tomcat starts, it creates a pool of worker threads. This pool sits idle, waiting for incoming connections.

When a new HTTP request arrives, Tomcat accepts the TCP connection on the acceptor thread, then hands the actual request processing off to a worker thread from the pool. From that moment forward, the entire lifecycle of that request — filters, controllers, services, database calls — happens on that one worker thread unless you explicitly change that.

This is called the thread-per-request model. It is the default behavior in Spring MVC.

// This entire method executes on one worker thread
@GetMapping("/orders/{id}")
public ResponseEntity<Order> getOrder(@PathVariable Long id) {
    return ResponseEntity.ok(orderService.findById(id));
}

The thread is occupied for the full duration of the request. If your method blocks on a database query for 200ms, the thread is held for those 200ms and cannot serve any other request during that time.

Async and Reactive Exceptions

If you introduce @Async, CompletableFuture, or Spring WebFlux, work may move to a different thread mid-request. When that happens, critical pieces of thread-local state — specifically the SecurityContextHolder and any active transaction — do not automatically follow. They are bound to the original thread and become invisible on the new thread.

@Async
public CompletableFuture<String> processAsync() {
    // This runs on a different thread
    // SecurityContextHolder.getContext() may return an empty context here
    Authentication auth = SecurityContextHolder.getContext().getAuthentication();
    return CompletableFuture.completedFuture(auth.getName()); // may throw NullPointerException
}

You must explicitly propagate state when crossing thread boundaries. Spring provides DelegatingSecurityContextRunnable and DelegatingSecurityContextCallable for this. In WebFlux, use ReactiveSecurityContextHolder.

Thread Pool and Request Concurrency

How the Thread Pool Works

Tomcat maintains a bounded thread pool. By default, it allows up to 200 worker threads. When a request arrives, a free thread is taken from the pool and assigned to that request. When the request finishes, the thread is returned to the pool and becomes available again.

You can tune this in application.properties:

server.tomcat.threads.max=200
server.tomcat.threads.min-spare=10
server.tomcat.accept-count=100

threads.max defines the maximum number of concurrent requests being actively processed. accept-count defines how many requests can be queued waiting for a thread when the pool is full.

Concurrent Request Processing

Two requests arriving at the same millisecond are handled on two separate threads simultaneously. They may be executing code in the same service class and the same repository class at the same time, but each has its own stack, its own local variables, and its own thread-local state. This is how Spring achieves request isolation without any locking overhead in the typical case.

Thread-47: GET /orders/101 -> SecurityContext(user=alice) -> Transaction-A -> Connection-1
Thread-52: GET /orders/202 -> SecurityContext(user=bob)   -> Transaction-B -> Connection-2

These two requests run in complete isolation from each other, even though they are executing the same code paths.

What Happens When the Pool is Full

When all 200 threads are occupied with in-progress requests, new incoming requests do not get rejected immediately. They are placed in a TCP accept queue (bounded by accept-count). If that queue also fills up, the server stops accepting new TCP connections, and clients will see a connection timeout or connection refused error.

This is the first hard throughput ceiling your application has.

Memory Usage of Request Threads

Thread Stack Size

Each thread has a dedicated stack allocated from JVM memory. In most JVMs, the default thread stack size is 512KB to 1MB. With 200 threads, that alone accounts for 100MB to 200MB of memory reserved for thread stacks, independent of heap usage.

You can tune stack size with the -Xss JVM flag:

java -Xss512k -jar myapp.jar

Reducing stack size allows more threads to exist within the same memory footprint, but too small a stack causes StackOverflowError for deeply recursive code.

Request and Response Objects

Every HTTP request creates a HttpServletRequest object on the heap. It holds the request headers, query parameters, path variables, and request body. The response object (HttpServletResponse) holds the response buffer. For large request bodies or file uploads, these objects can consume significant heap memory.

Spring also creates objects throughout the chain: filter instances (shared, not per-request), the Authentication object in the security context, the @Transactional proxy interceptor call frames, repository proxies, and finally the response DTO serialized to JSON.

All of these are garbage collected after the request completes, but during the request they all live on the heap. High-traffic applications can generate significant GC pressure if response objects are large or if many requests are running concurrently.

Security Processing and User Isolation

The Filter Chain

Before a request ever reaches DispatcherServlet, it passes through the Servlet filter chain. Spring Security registers itself as a filter in this chain, specifically through FilterChainProxy. This proxy delegates to a series of security filters, each with a specific responsibility.

The most important for most applications is UsernamePasswordAuthenticationFilter or, in stateless APIs, a custom JWT authentication filter. This filter extracts the credentials or token from the request, validates them, and creates an Authentication object.

SecurityContextHolder and ThreadLocal Isolation

Once authenticated, Spring Security stores the Authentication object in the SecurityContextHolder. Under the hood, SecurityContextHolder uses a ThreadLocal variable to store the security context.

A ThreadLocal is a per-thread variable. Each thread has its own independent copy. When thread-47 sets a value in a ThreadLocal, thread-52 cannot read it. This is precisely what guarantees that user Alice's authentication on thread-47 is never visible on thread-52, which is serving user Bob.

// Spring does this on your behalf during the filter chain
SecurityContextHolder.getContext().setAuthentication(authentication);

// Your code can then safely read the authenticated user anywhere in the same request
Authentication auth = SecurityContextHolder.getContext().getAuthentication();
String username = auth.getName();

After the request completes, the SecurityContextPersistenceFilter clears the security context from the thread before it returns to the pool. If this did not happen, the next request handled by the same thread would inherit the previous user's authentication — a serious security bug.

[Image Placeholder: Spring Security filter chain showing ThreadLocal security context isolation per request thread] Image Generation Prompt: "clean minimal technical diagram showing Spring Security filter chain with ThreadLocal security context isolation, two parallel request threads each having their own security context with different authenticated users, educational software architecture diagram, white background"

Async and Reactive Exceptions

In asynchronous processing, the thread changes. When a task is submitted to a thread pool via @Async or CompletableFuture.supplyAsync(), the new thread has no security context. SecurityContextHolder.getContext().getAuthentication() returns null.

To propagate the security context across threads, wrap your runnable:

// Manually propagate security context to async thread
SecurityContext context = SecurityContextHolder.getContext();
executor.submit(new DelegatingSecurityContextRunnable(() -> {
    // SecurityContext is now available here
    String user = SecurityContextHolder.getContext().getAuthentication().getName();
}, context));

In Spring WebFlux (reactive), there is no thread-local model at all. Use ReactiveSecurityContextHolder:

return ReactiveSecurityContextHolder.getContext()
    .map(ctx -> ctx.getAuthentication().getName())
    .flatMap(username -> userService.findByUsername(username));

Request Routing Inside Spring MVC

DispatcherServlet

After the filter chain completes, the request reaches DispatcherServlet. This is the central front controller in Spring MVC. There is one DispatcherServlet per application context, and every request passes through it.

DispatcherServlet does not contain any business logic. Its job is routing. It consults a list of HandlerMapping implementations to find which controller method should handle this specific request.

Handler Mappings

The primary handler mapping for annotation-based controllers is RequestMappingHandlerMapping. During application startup, it scans all beans annotated with @Controller or @RestController and builds a map of URL patterns to method references.

When a request for GET /orders/101 arrives, RequestMappingHandlerMapping looks up this URL and HTTP method combination, finds the matching @GetMapping("/orders/{id}") method, and returns a HandlerMethod object wrapping that method reference.

DispatcherServlet then finds the appropriate HandlerAdapter (usually RequestMappingHandlerAdapter) and invokes the method. The adapter handles argument resolution (reading @PathVariable, @RequestBody, @RequestParam) and return value handling (writing the response).

@RestController
@RequestMapping("/orders")
public class OrderController {

    @GetMapping("/{id}")
    public Order getOrder(@PathVariable Long id) {
        // DispatcherServlet resolved this method via RequestMappingHandlerMapping
        // RequestMappingHandlerAdapter resolved @PathVariable from the URL
        return orderService.findById(id);
    }
}

Application Layer Execution

The Layered Architecture

Spring Boot applications typically follow a three-layer architecture: controller, service, and repository. This is not a framework requirement but a widely adopted convention that separates HTTP concerns from business logic from data access.

The controller receives the HTTP request and translates it into method calls. It should contain no business logic. The service layer contains the domain logic. The repository layer handles database interaction, typically through Spring Data JPA.

How the Call Flows Through the Layers

From the developer's perspective, this is a straightforward method call chain. But Spring wraps each layer with proxy objects that add behavior transparently.

When your controller calls orderService.findById(id), it is not calling your OrderServiceImpl directly. It is calling a Spring-generated proxy. That proxy checks whether any AOP advice applies to this method — including @Transactional transaction management, caching interceptors, security authorization checks, and any custom aspects you may have defined.

@Service
public class OrderServiceImpl implements OrderService {

    @Autowired
    private OrderRepository orderRepository;

    @Transactional(readOnly = true)
    public Order findById(Long id) {
        // The proxy opens a transaction before this line executes
        return orderRepository.findById(id)
            .orElseThrow(() -> new OrderNotFoundException(id));
        // The proxy commits/closes the transaction after this returns
    }
}

The OrderRepository is itself a JPA repository proxy that translates the findById call into a JPQL query, manages the EntityManager, and handles the result mapping. All of this happens within the same worker thread.

Transaction Lifecycle

How Transactions Begin

When a method annotated with @Transactional is called through its proxy, Spring's TransactionInterceptor runs before the method body executes. It checks the transaction propagation setting (default is REQUIRED, meaning use an existing transaction or create a new one) and, if a new transaction is needed, calls the PlatformTransactionManager to begin one.

For JPA with HikariCP, beginning a transaction means: obtain a database connection from the connection pool, set autoCommit=false on that connection, and bind both the connection and the transaction status to the current thread using TransactionSynchronizationManager.

// TransactionSynchronizationManager stores the connection in a ThreadLocal map
// keyed by the DataSource
resources.set(dataSource, connectionHolder); // stored in ThreadLocal

This is the key point: the transaction is thread-bound. The database connection holding the open transaction is stored in a ThreadLocal, which means it is only accessible from the thread that started the transaction.

Commit and Rollback

After your method returns normally, the TransactionInterceptor calls commit() on the transaction. The connection flushes any pending SQL, commits the transaction on the database, sets autoCommit=true, and returns the connection to the pool.

If your method throws an unchecked exception (a RuntimeException or Error), the interceptor calls rollback() instead. The in-progress transaction is rolled back on the database, and the connection is returned to the pool.

@Transactional
public void placeOrder(OrderRequest request) {
    Order order = new Order(request);
    orderRepository.save(order);       // INSERT executed but not yet committed
    inventoryService.deduct(request);  // if this throws, the INSERT is rolled back
    // commit happens here if no exception was thrown
}

Async and Reactive Exceptions

Transactions are bound to the current thread via ThreadLocal. If the thread changes mid-transaction — which happens with @Async, CompletableFuture, or reactive pipelines — the new thread has no transaction context. Any JPA operations on the new thread will either start a new transaction or run without one, depending on propagation settings.

There is no clean way to continue a JDBC transaction across thread boundaries. The recommended pattern is to complete all transactional work on a single thread, then hand off non-transactional work asynchronously.

For reactive applications, Spring provides ReactiveTransactionManager and @Transactional support in WebFlux, but this operates on a Reactor context rather than ThreadLocal, and requires reactive-compatible database drivers (R2DBC, not JDBC).

Database Connection Pool Usage

How the Pool Works

Spring Boot auto-configures HikariCP as the default connection pool. On startup, HikariCP creates a pool of physical JDBC connections to your database server. By default, maximumPoolSize is 10.

When a transaction begins (as described above), the TransactionInterceptor asks HikariCP for a connection. HikariCP checks if any connections are idle in the pool. If one is available, it marks it as borrowed and returns it. If none are available, the calling thread waits up to connectionTimeout milliseconds (default 30 seconds) for a connection to become free.

spring.datasource.hikari.maximum-pool-size=10
spring.datasource.hikari.minimum-idle=5
spring.datasource.hikari.connection-timeout=30000
spring.datasource.hikari.idle-timeout=600000

The Connection Is Held for the Transaction Duration

A connection is held for the entire duration of the transaction. If your transaction opens at the start of a service method and does not commit until the end, the connection is tied up for that entire period — including any time spent doing computation, calling external services, or executing slow queries.

This is the second hard throughput ceiling: with maximumPoolSize=10, only 10 transactions can hold active database connections at any given moment. Thread 11 through 200 will be blocked waiting for a connection if the first 10 threads are all in active transactions.

Pool Exhaustion

When the connection pool is exhausted and no connection becomes free within connectionTimeout, HikariCP throws a SQLTransientConnectionException. This surfaces in your application as a 500 error.

The common mistake here is setting maximumPoolSize too high. More connections do not mean more throughput; they mean more concurrency on the database, which eventually saturates database CPU. The right pool size depends on your database server's capacity and your query patterns.

Response Generation

Returning from the Controller

Once the service method returns its result, control flows back through the proxy layers (any @AfterReturning advice, transaction commit) and eventually back to DispatcherServlet, which now has the return value from the controller method.

For @RestController methods, the return value needs to be serialized to an HTTP response body. DispatcherServlet hands this off to a HandlerMethodReturnValueHandler, which for ResponseEntity or plain objects, delegates to an HttpMessageConverter.

Jackson and JSON Serialization

Spring Boot auto-configures Jackson's MappingJackson2HttpMessageConverter. When the content negotiation process determines that the client expects application/json (typically from the Accept header or by default), Jackson is used to serialize the return object.

Jackson uses reflection to inspect the object's fields, calls getter methods, and writes a JSON string to the HttpServletResponse's output stream.

// Your controller returns this
return new OrderResponse(order.getId(), order.getStatus(), order.getTotal());

// Jackson serializes it to:
// {"id": 101, "status": "CONFIRMED", "total": 49.99}
// and writes it to the response output buffer

The response is then flushed over the TCP connection back to the client, and the worker thread is released back to Tomcat's thread pool.

System Limits and Throughput

The Three Hard Limits

Every Spring Boot application operating under the thread-per-request model has three fundamental throughput ceilings:

The first is the thread pool size. With a default of 200 threads, you can process at most 200 requests concurrently. If each request takes 100ms on average, your theoretical maximum throughput is around 2000 requests per second — and in practice, it will be lower due to overhead.

The second is the database connection pool size. With a default of 10 connections, at most 10 requests can be in an active transaction simultaneously. If your average transaction time is 20ms, that is 500 transactional operations per second before threads start queuing for connections.

The third is the database server itself — its CPU, memory, I/O, and the complexity of your queries. The connection pool is a client-side throttle that protects the database. If you bypass it by increasing pool size too aggressively, you transfer the bottleneck to the database server instead.

Max concurrent requests:        server.tomcat.threads.max = 200
Max concurrent DB transactions: spring.datasource.hikari.maximum-pool-size = 10
Queue for waiting requests:     server.tomcat.accept-count = 100
Wait for DB connection:         spring.datasource.hikari.connection-timeout = 30000ms

Throughput Formula

Effective throughput depends on request latency. If average request duration is L milliseconds and you have T threads, your maximum throughput is approximately T / L * 1000 requests per second.

A 50ms average request latency with 200 threads gives roughly 4000 req/sec theoretical maximum — but that only holds if no other bottleneck intervenes, which in practice, the database connection pool usually does first.

Bottlenecks and Failure Scenarios

Thread Exhaustion

Thread exhaustion occurs when all 200 worker threads are occupied and the TCP accept queue is also full. New clients receive connection refused errors. Existing requests in the accept queue receive connection timeout errors after waiting.

The most common cause is not raw traffic volume — it is slow requests. If your application normally handles requests in 50ms but a downstream service starts responding in 5 seconds, threads start piling up. With 200 threads and a 5-second average duration, the system saturates at just 40 requests per second instead of 4000.

Symptoms include rising response time metrics, thread pool exhaustion alerts, and eventually connection refused errors from the load balancer.

The solution is a combination of: timeouts on external calls, circuit breakers, and either horizontal scaling or moving to a reactive model for I/O-heavy workloads.

Connection Pool Exhaustion

Connection pool exhaustion happens when all 10 (or however many you configured) database connections are held and no new one becomes available within connectionTimeout. Threads that need a connection block, which accelerates thread exhaustion.

The common triggers are: slow database queries holding connections longer than usual, unexpectedly high traffic, a transaction that holds a connection open across a remote API call (a very common mistake), or a connection leak where connections are borrowed but never returned.

// Common mistake: transaction spans an external HTTP call
@Transactional
public OrderConfirmation checkout(CartRequest cart) {
    Order order = orderRepository.save(new Order(cart));     // borrows connection
    paymentResult = paymentGateway.charge(cart.getPayment()); // holds connection for 2-3 seconds
    order.setStatus(paymentResult.getStatus());
    return orderRepository.save(order);                      // releases connection after this
}

The fix is to restructure so that the remote call happens outside the transaction, and the transaction only wraps the actual database operations.

Request Timeouts

If Tomcat has no explicit request timeout configured and a slow request holds a thread for minutes, it simply occupies that thread for the full duration. You can set a read timeout:

server.tomcat.connection-timeout=20000

At the application level, setting timeouts on RestTemplate, WebClient, and database query execution limits prevents single slow operations from consuming thread resources indefinitely.

Final Visualization — Complete Request Lifecycle

The diagram below ties together every stage described in this article. Following the full path of a single request reveals how each component depends on the one before it, and why bottlenecks in one layer cascade into failures across the rest.

The sequence for a single request, fully detailed:

Step 1 — TCP Acceptance. The client opens a TCP connection. Tomcat's acceptor thread receives it and places it in the worker queue.

Step 2 — Thread Assignment. A worker thread from the thread pool picks up the request. This thread will carry the request from this point until the response is written.

Step 3 — Filter Chain. The request traverses the filter chain. Spring Security filters extract the JWT or session token, validate it, create an Authentication object, and store it in the SecurityContextHolder ThreadLocal on this thread.

Step 4 — DispatcherServlet Routing. DispatcherServlet consults RequestMappingHandlerMapping, finds the target controller method, and invokes it via RequestMappingHandlerAdapter.

Step 5 — Controller Proxy. The controller calls the service through a Spring AOP proxy. The proxy evaluates any applicable aspects before delegating to the actual service method.

Step 6 — Transaction Begins. The @Transactional proxy intercepts the service method call. TransactionInterceptor asks HikariCP for a connection. HikariCP lends one from the pool. The connection is bound to the current thread via TransactionSynchronizationManager.

Step 7 — Repository and Query Execution. The repository proxy translates the method call into SQL via JPA/Hibernate. The SQL is executed on the borrowed connection. Results are mapped to entity objects.

Step 8 — Transaction Commits. Control returns to the @Transactional proxy. No exception was thrown, so the transaction is committed. The connection is returned to HikariCP's pool.

Step 9 — Response Serialization. The controller returns a response object. Jackson serializes it to JSON and writes it to the HttpServletResponse output buffer.

Step 10 — Security Context Cleared. Spring Security's SecurityContextPersistenceFilter clears the ThreadLocal context, ensuring no user data leaks to the next request.

Step 11 — Thread Released. The response is flushed over TCP. The worker thread is returned to Tomcat's thread pool, ready for the next request.

Conclusion

Understanding the internal lifecycle of a Spring Boot request is not just academic—it provides a mental model that helps you reason about performance, debug production issues, and make confident architectural decisions.

The thread-per-request model in Spring MVC ensures that every layer of your application executes on the same OS thread. This thread carries all state: security context, transaction bindings, and borrowed database connections. Any break in that continuity—through async execution, reactive pipelines, or additional thread pools—requires explicit management of this state.

Your application's throughput ceiling isn’t determined by your code alone. It’s defined by three key numbers: threads.max, maximum-pool-size, and average request latency. Measure these under realistic load and tune them for your actual workload instead of relying on defaults.

When traffic spikes, failures follow a predictable cascade: slow requests fill threads, threads waiting for connections queue up, the connection pool exhausts, and clients see 500 errors. Understanding this pattern allows you to detect issues early and respond before a full outage occurs.

If you find this guide useful, share it with your friends!

Docker has revolutionized software development by enabling consistent, isolated application environments through containerization. This cheat sheet provides a concise reference for developers, students, and DevOps beginners to quickly master essential Docker concepts and commands. Use it as your go-to guide for daily Docker tasks, from building images to orchestrating multi-container applications.

Introduction to Docker

What is Docker and Why Use It

Docker is a platform that packages applications and their dependencies into lightweight containers that run consistently across any system. It eliminates the "works on my machine" problem by ensuring identical environments from development to production. Developers use Docker to simplify dependency management, streamline deployment, and improve resource utilization compared to traditional virtual machines.

Problems Docker Solves

• Environment inconsistencies between development, testing, and production
• Complex dependency management and version conflicts
• Inefficient resource utilization with traditional virtualization
• Slow onboarding of new team members to existing projects
• Difficulty scaling applications across different infrastructure

Docker vs Virtual Machines

Containers share the host operating system kernel and run as isolated processes, while virtual machines include complete guest operating systems. This fundamental difference makes containers more lightweight, faster to start, and more resource-efficient than VMs.

What is a Container

A container is a standard unit of software that packages code and all its dependencies so the application runs quickly and reliably across computing environments. Containers virtualize the operating system rather than hardware, making them portable and efficient. Each container runs as an isolated process in user space on the host operating system.

Docker Architecture

Docker uses a client-server architecture where the client communicates with a daemon that handles building, running, and distributing containers. The daemon exposes a REST API that clients use to manage containers, images, networks, and volumes.

Key Components:

• Docker Client: Command-line tool that sends commands to the daemon
• Docker Daemon: Background service that manages Docker objects
• Docker Images: Read-only templates used to create containers
• Docker Containers: Runnable instances of Docker images
• Docker Registry: Repository for storing and distributing images

Docker Installation

Installing Docker on Windows

Download Docker Desktop for Windows from the official Docker Hub website and run the installer. Enable WSL 2 integration during installation for better performance and Linux compatibility. After installation, Docker Desktop runs as a system tray application.

Installing Docker on Mac

Download Docker Desktop for Mac from Docker Hub and drag the application to your Applications folder. The installer includes both Intel and Apple Silicon versions for compatibility. Docker Desktop runs as a menu bar application with a simple management interface.

Installing Docker on Linux

Use the official package manager for your distribution to install Docker Engine. Add your user to the docker group to run commands without sudo. Enable the Docker service to start automatically on system boot.

# Verify Docker installation
docker --version
docker run hello-world

Docker Core Concepts

Docker Images

An image is a read-only template with instructions for creating a Docker container. Images are built from Dockerfiles and consist of multiple layers that can be cached and reused. You can create custom images or use pre-built images from Docker Hub.

Docker Containers

A container is a runnable instance of an image that includes the application, its dependencies, and runtime configuration. Containers can be started, stopped, moved, and deleted while maintaining their state through volumes. Multiple containers can run from the same image with different configurations.

Container Lifecycle

Containers transition through different states during their lifetime, from creation to removal. Understanding these states helps in managing container resources and debugging issues.

Dockerfile

A Dockerfile is a text file containing instructions to build a Docker image automatically. Each instruction creates a layer in the image, with later layers building on previous ones. The Dockerfile defines the base image, application code, dependencies, and startup commands.

FROM node:18-alpine

WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production

COPY . .

EXPOSE 3000
USER node
CMD ["node", "server.js"]

Docker Image Layers

Each instruction in a Dockerfile creates a new layer in the final image, forming a stack of read-only filesystem changes. Docker caches these layers, so unchanged instructions reuse existing layers during subsequent builds. Layer caching significantly speeds up build times and reduces bandwidth when sharing images.

Working with Docker Images

Building Docker Images

The docker build command creates images from a Dockerfile and context directory. Tags help organize and version images for easy reference and distribution. Build context includes all files sent to the daemon during image creation.

docker build -t my-api:latest .
docker build -t my-api:v1.0 --build-arg ENV=production .

Managing Images

Images require regular maintenance to remove unused ones and free disk space. Local images are stored in the Docker daemon's storage area and can be inspected for details.

docker images
docker image ls --filter dangling=true
docker rmi my-api:latest
docker image prune -a
docker inspect my-api

Working with Containers

Running Containers

Containers start from images and can be configured with ports, volumes, environment variables, and restart policies. Port publishing maps container ports to host ports for external access. Background mode (-d) runs containers in the background, while interactive mode (-it) provides terminal access.

docker run -d --name web-app -p 8080:80 nginx:alpine
docker run --rm -it -v $(pwd):/app node:18 npm init
docker run --env DB_HOST=localhost --env DB_PORT=5432 my-api

Managing Containers

Container management involves monitoring, stopping, starting, and removing containers as needed. Container IDs can be referenced using the first few characters of the full ID.

docker ps -a
docker stop web-app
docker start web-app
docker restart web-app
docker rm web-app
docker logs --tail 50 web-app

Interactive Containers

Interactive containers provide shell access for debugging, development, and exploration. The -it flag combines interactive mode with a pseudo-TTY for terminal support. You can execute additional commands in running containers with docker exec.

docker run -it --name debug ubuntu:22.04 bash
docker exec -it web-app sh
docker run -it --entrypoint sh alpine:latest

Docker Volumes and Storage

Docker Volumes

Volumes are the preferred mechanism for persisting data generated by and used by Docker containers. They are completely managed by Docker and stored in the Docker storage directory. Volumes can be named for easy reference and shared between multiple containers.

docker volume create app-data
docker volume ls
docker run -v app-data:/var/lib/postgresql/data postgres:15
docker volume inspect app-data

Bind Mounts

Bind mounts map a host file or directory into a container, providing direct access to host filesystem. They are ideal for development where code changes should reflect immediately in the container. Bind mounts depend on the host filesystem structure and permissions.

docker run -v /host/data:/container/data nginx
docker run --mount type=bind,src=/host/config,target=/app/config nginx

tmpfs Mounts

tmpfs mounts store data in the host's memory only, never written to the filesystem. They provide high-performance temporary storage for sensitive data that shouldn't persist. tmpfs mounts are ideal for secrets, tokens, or cache data.

docker run --tmpfs /app/cache:rw,noexec,nosuid,size=64m nginx

Storage Types Comparison

Docker Networking

Docker Networking Basics

Networking enables containers to communicate with each other and with the outside world. Each container gets a virtual network interface with an IP address when created. Docker provides pluggable network drivers for different communication patterns.

docker network create --driver bridge my-network
docker network connect my-network my-container
docker network disconnect my-network my-container
docker network inspect bridge

Network Types

Docker supports multiple network drivers to address different communication requirements. The choice of network type affects container isolation, performance, and accessibility.

Docker Registry

Docker Hub

Docker Hub is the default public registry where you can find thousands of official and community images. It provides automated builds, webhooks, and organization accounts for teams. Official images are maintained by Docker or the software vendor, ensuring quality and security.

Private Registries

Private registries store images internally for security, compliance, or performance reasons. You can run your own registry using the registry:2 image or use cloud provider solutions. Private registries require authentication and can be integrated with CI/CD pipelines.

Pushing Images

Pushing images makes them available to other developers and deployment environments. Images must be tagged with the registry name before pushing. Authentication is required for private registries and Docker Hub.

docker login -u username
docker tag my-app:latest username/my-app:latest
docker push username/my-app:latest
docker push myregistry.com:5000/my-app:latest

Pulling Images

Pulling downloads images from a registry to your local Docker daemon. Images are pulled by name and tag, with :latest being the default if no tag specified. Pulls only download layers not already present locally.

docker pull nginx:alpine
docker pull myregistry.com:5000/my-api:v2
docker pull postgres:15

Multi Container Applications

Problems with Single Containers

Single containers become complex when applications require databases, caches, or multiple services. Managing dependencies, networking, and data persistence across services becomes challenging. Scaling individual components independently is impossible with monolithic containers.

Multi Container Architecture

Multi-container applications separate concerns into specialized containers that communicate over networks. Each container runs a single process or service, following the Unix philosophy. This architecture enables independent scaling, updates, and maintenance of each component.

Docker Compose

What is Docker Compose

Docker Compose is a tool for defining and running multi-container Docker applications using YAML files. It simplifies container orchestration by managing networks, volumes, and service dependencies. One command starts all services defined in the compose file.

docker-compose.yml Structure

The compose file defines services, networks, and volumes for your application. Services specify container images, ports, environment variables, and dependencies. Networks enable service discovery using service names as hostnames.

version: '3.8'

services:
  web:
    build: ./web
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
    depends_on:
      - api

  api:
    build: ./api
    environment:
      - DB_HOST=postgres
    depends_on:
      postgres:
        condition: service_healthy

  postgres:
    image: postgres:15-alpine
    volumes:
      - pg-data:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=${DB_PASSWORD}
    healthcheck:
      test: ["CMD-SHELL", "pg_isready"]

volumes:
  pg-data:

Running Compose Applications

Compose commands manage the entire application lifecycle from a single directory. Services start in dependency order and can be scaled for specific services. Compose projects are isolated by directory or project name.

docker compose up -d
docker compose down -v
docker compose logs -f
docker compose ps
docker compose exec web sh
docker compose restart api

Compose with Networks

Custom networks provide service isolation and controlled communication between containers. Networks can be defined with specific drivers and configurations. Services on the same network can resolve each other by service name.

version: '3.8'

services:
  frontend:
    build: ./frontend
    networks:
      - frontend
      - backend

  backend:
    build: ./backend
    networks:
      - backend
    depends_on:
      - redis

  redis:
    image: redis:alpine
    networks:
      - backend

networks:
  frontend:
  backend:
    driver: bridge

Compose with Volumes

Volumes in Compose persist data across container restarts and recreations. Named volumes are managed by Docker, while bind mounts map host directories. Volume configurations can include driver options and labels.

version: '3.8'

services:
  mysql:
    image: mysql:8
    volumes:
      - mysql-data:/var/lib/mysql
      - ./backup:/backup:ro
    environment:
      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}

  app:
    build: .
    volumes:
      - ./uploads:/app/uploads
      - ./config:/app/config:ro

volumes:
  mysql-data:
    driver: local
    driver_opts:
      type: none
      device: /data/mysql
      o: bind

Compose with Port Binding

Port binding exposes container ports to the host or other networks. Published ports can be mapped to specific host ports or random high ports. Port configurations support both IPv4 and IPv6 addresses.

version: '3.8'

services:
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
      - "127.0.0.1:8080:80"

  app:
    build: .
    ports:
      - "3000"
    expose:
      - "3000"

Useful Docker Commands Cheat Sheet

Docker Best Practices

Small Images

• Use minimal base images like alpine, slim variants, or distroless
• Combine RUN commands to reduce layer count
• Remove package manager caches and temporary files
• Use multi-stage builds to discard build dependencies

Multi-Stage Builds

• Separate build environment from runtime environment
• Copy only artifacts from build stage to final stage
• Reduce final image size by excluding build tools
• Maintain separate stages for development and production

FROM node:18 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM nginx:alpine
COPY --from=builder /app/dist /usr/share/nginx/html
COPY nginx.conf /etc/nginx/nginx.conf

.dockerignore

• Exclude version control directories (.git, .svn)
• Ignore local configuration files (.env, .local)
• Skip build artifacts and dependency caches
• Exclude test files and documentation

.git
node_modules
.env
*.log
Dockerfile
.dockerignore

Non-Root Containers

• Create and use non-root users in containers
• Set USER instruction in Dockerfile
• Avoid running processes as root
• Use read-only root filesystems when possible

Docker Workflow

The Docker development workflow provides a consistent path from code to deployment. Changes are tested in containers matching production environments. Images are versioned and stored in registries for distribution.

1. Write application code and create Dockerfile
2. Build image with docker build and tag appropriately
3. Test locally with docker run or docker compose
4. Push image to registry with docker push
5. Deploy to production using docker run or orchestrator
6. Monitor and update containers as needed

Conclusion

Docker has become the industry standard for containerization, enabling developers to build, ship, and run applications anywhere with consistency and efficiency. By packaging applications with their dependencies, containers eliminate environment inconsistencies and streamline the development-to-production pipeline. This cheat sheet provides essential commands and concepts for daily Docker use, from basic container operations to multi-application orchestration with Compose. Keep it handy as your quick reference to master container workflows and build portable, scalable applications in any environment.

Git is the industry-standard version control system that tracks changes in your code, enables collaboration, and protects your project history. GitHub builds on Git by providing cloud hosting, pull requests, and team collaboration tools. This cheat sheet gives you quick access to essential Git commands, workflows, and best practices without the fluff.

Getting Started with Git

What is Git and Why Use It

Git is a distributed version control system that records snapshots of your files, allowing you to revert to previous states, compare changes, and collaborate without overwriting work. Unlike centralized systems, every developer has a complete copy of the project history locally. Git was created by Linus Torvalds in 2005 to manage the Linux kernel development efficiently.

Key Points

• Git tracks content changes, not just file names
• Every clone is a full backup of the entire repository
• Branching in Git is lightweight and inexpensive
• Changes are stored as snapshots, not file differences
• Git uses cryptographic hashing (SHA-1) to ensure data integrity

Local vs Remote Repositories

A local repository exists on your machine and contains the complete project history with all branches. A remote repository is hosted on a server like GitHub, GitLab, or Bitbucket, enabling team collaboration and backup.

Key Points

• Local repos allow offline work and fast operations
• Remote repos serve as the centralized source of truth
• Changes move between repos through push and pull operations
• You can connect multiple remotes to a single local repo
• The default remote name is typically "origin"

Git Installation and Configuration

Install Git from git-scm.com or using your package manager. After installation, configure your identity and preferences to ensure commits are properly attributed.

# Set your global username and email
git config --global user.name "Your Name"
git config --global user.email "your.email@example.com"

# Set your default branch name (Git 2.28+)
git config --global init.defaultBranch main

# Configure line endings (Windows vs Mac/Linux)
git config --global core.autocrlf input  # Mac/Linux
git config --global core.autocrlf true   # Windows

# View all configurations
git config --list

Initializing and Cloning Repositories

Start version control in an existing project with git init or copy an existing remote repository with git clone. These are the entry points to any Git workflow.

# Initialize a new repository in current folder
git init

# Initialize in a specific directory
git init project-name

# Clone an existing repository
git clone <https://github.com/user/repo.git>

# Clone into a specific folder name
git clone <https://github.com/user/repo.git> my-folder

# Clone with SSH (recommended for frequent pushes)
git clone git@github.com:user/repo.git

Basic Git Workflow

The fundamental workflow moves changes from your working directory to the staging area, then commits them to the repository history. This three-step process gives you control over what gets recorded.

# Check status of your working directory
git status

# Add specific files to staging
git add filename.txt

# Add all changes in current directory
git add .

# Add all changes in the entire repository
git add -A

# Commit staged changes with a message
git commit -m "Descriptive commit message"

# View commit history
git log

# View condensed one-line history
git log --oneline

# See what's changed but not staged
git diff

# See staged changes
git diff --staged

Key Points

• Working directory contains files you're editing
• Staging area (index) is where you prepare your next commit
• Commits are permanent snapshots with unique IDs
• Always write clear commit messages explaining why, not what

Ignoring Files with .gitignore

The .gitignore file tells Git which files or patterns to intentionally untrack, keeping your repository clean of temporary files, build artifacts, and sensitive information.

Key Points

• Place .gitignore in your repository root or subdirectories
• Patterns support wildcards and negation with !
• Already tracked files are not affected by ignore rules
• Use global gitignore for OS-specific files
• Common ignores: node_modules, .env, .DS_Store, build folders

# Example .gitignore patterns
# Dependencies
node_modules/
vendor/

# Build outputs
dist/
build/
*.exe
*.dll

# Environment files
.env
.env.local

# IDE files
.vscode/
.idea/
*.swp

# OS files
.DS_Store
Thumbs.db

Branching and Collaboration

Branching in Git

Branches are lightweight pointers to specific commits, allowing parallel development without affecting the main codebase. Creating and switching branches is instantaneous because Git just moves a pointer.

# List local branches
git branch

# List all branches including remote
git branch -a

# Create a new branch
git branch feature-login

# Switch to a branch
git checkout feature-login

# Create and switch in one command
git checkout -b feature-login

# Rename a branch
git branch -m old-name new-name

# Delete a branch (fully merged)
git branch -d feature-login

# Force delete unmerged branch
git branch -D feature-login

Key Points

• Main branch is typically called main or master
• Keep branches focused on single features or fixes
• Delete branches after merging to keep clean history
• Branch names should be descriptive: feature/, bugfix/, hotfix/

Handling Merge Conflicts

A merge conflict occurs when Git cannot automatically reconcile differences between two branches. This usually happens when the same line of a file has been modified differently in two branches.

Signs of a Conflict

• Git shows messages like CONFLICT (content): Merge conflict in <file>
• git status shows files as both modified

Resolving Conflicts

1. Open the conflicted file(s). Git marks conflicts like this:

<<<<<<< HEAD
Your changes here
=======
Incoming changes here
>>>>>>> feature-branch

1. Edit the file to keep the correct version or combine changes.
2. After fixing conflicts, mark as resolved:

git add filename.txt

1. Complete the merge:

git commit
# Or if rebasing
git rebase--continue

Best Practices

• Pull frequently to minimize conflicts
• Communicate with teammates about overlapping work
• Keep branches small and focused
• Use tools like git mergetool for complex merges

Merging vs Rebasing

Both commands integrate changes from one branch into another, but they handle history differently. Merges preserve exact chronology while rebasing creates a linear history.

# Merge feature into main (stay on main branch)
git merge feature-branch

# Merge with no fast-forward (always creates merge commit)
git merge --no-ff feature-branch

# Rebase current branch onto main
git checkout feature-branch
git rebase main

# Interactive rebase (squash, edit, reorder commits)
git rebase -i HEAD~3

Key Points

• Never rebase public/shared branches
• Merge when preserving context matters
• Rebase for clean, linear history before pull requests
• Interactive rebase helps clean up local commits

Remote Repositories

Remotes are versions of your repository hosted on servers. They enable collaboration by allowing you to share commits and pull changes from teammates.

# Add a remote repository
git remote add origin <https://github.com/user/repo.git>

# List configured remotes
git remote -v

# Fetch changes without merging
git fetch origin

# Fetch and merge (pull) from remote
git pull origin main

# Push local commits to remote
git push origin main

# Push and set upstream tracking
git push -u origin main

# Remove a remote
git remote remove origin

# Rename a remote
git remote rename origin upstream

Key Points

• origin is the default name for your cloned remote
• upstream often refers to the original repository in forks
• Fetch is safe; pull combines fetch and merge
• Always pull before pushing to avoid conflicts

Collaborating with GitHub

GitHub transforms Git from a version control system into a complete collaboration platform. It provides the interface and tools that enable teams to review code, track work, and automate workflows around your Git repositories.

Key Points

• Pull requests are the core of code review and collaboration
• Issues track bugs, feature requests, and tasks
• Projects visualize work using Kanban boards
• GitHub Actions automate testing and deployment
• Code owners can be automatically requested for reviews

Pull Request Workflow

1. Push your feature branch to GitHub
2. Open a pull request comparing your branch to main
3. Add a clear description and request reviewers
4. Discuss changes and push additional commits if needed
5. Merge once approved and all status checks pass
6. Delete the branch after merging to keep the repository clean

Forking and Syncing with Upstream

Forking creates a personal copy of someone else's repository on GitHub. This is essential for open-source contributions where you don't have write access.

Key Points

• Forks are independent repositories under your account
• Keep your fork synced to contribute clean pull requests
• The original repo is typically called "upstream"
• Never commit directly to upstream main

# Add original repository as upstream remote
git remote add upstream <https://github.com/original/repo.git>

# Fetch upstream changes
git fetch upstream

# Checkout your local main
git checkout main

# Merge upstream changes into your main
git merge upstream/main

# Push updates to your fork
git push origin main

Viewing Commit History

Git's log provides powerful filtering and formatting options to navigate project history. Understanding history helps debug issues and understand code evolution.

# Standard log with details
git log

# One line per commit
git log --oneline

# Graph view with branches
git log --graph --oneline --all

# Show last 3 commits
git log -3

# Search commits by message
git log --grep="fix bug"

# Show commits by author
git log --author="username"

# Show files changed in each commit
git log --stat

# View specific file history
git log -p filename.txt

Key Points

• Each commit has a unique 40-character SHA hash
• Use -oneline for the 7-character short hash
• Combine flags for custom views
• git show displays a single commit's details

Undoing Changes

Git provides multiple ways to undo changes depending on what stage they're in and whether you've shared them. Understanding the difference between reset, revert, and checkout is crucial.

# Unstage a file (keep changes)
git reset HEAD filename.txt

# Discard working directory changes
git checkout -- filename.txt

# Restore file to previous commit state
git restore filename.txt

# Undo last commit, keep changes staged
git reset --soft HEAD~1

# Undo last commit, unstage changes
git reset HEAD~1

# Undo last commit, discard all changes
git reset --hard HEAD~1

# Create new commit that undoes previous commit
git revert HEAD

# Revert a specific commit by hash
git revert abc123

Key Points

• reset moves the branch pointer and optionally modifies working directory
• revert creates a new commit that undoes changes (safe for shared history)
• checkout or restore for working directory files
• Never use -hard if you're unsure about losing changes

Tagging and Releases

Tags mark specific points in history as important, typically for releases. They're like branch references that never move.

# Create lightweight tag
git tag v1.0.0

# Create annotated tag
git tag -a v1.0.0 -m "Release version 1.0.0"

# List all tags
git tag

# List tags matching pattern
git tag -l "v1.*"

# Push specific tag to remote
git push origin v1.0.0

# Push all tags
git push --tags

# Delete local tag
git tag -d v1.0.0

# Delete remote tag
git push origin --delete v1.0.0

# Checkout tag (detached HEAD)
git checkout v1.0.0

Collaboration Best Practices

Effective collaboration requires clear conventions and communication. These practices prevent conflicts and keep history readable.

Key Points

• Write commit messages in imperative tense: "Fix login bug"
• Pull before starting work to stay updated
• Keep commits focused on single changes
• Use feature branches for all changes
• Review your own pull request before requesting reviews
• Resolve conflicts locally before pushing
• Communicate breaking changes in commit messages

Branching Strategies

Different workflows suit different team sizes and release cycles. Choose the strategy that matches your deployment frequency and team structure.

Git Flow

• main always reflects production-ready state
• develop integrates completed features
• Release branches prepare for production
• Hotfix branches patch production directly

GitHub Flow

• Anything in main is deployable
• Create branches for features/fixes
• Open pull requests for discussion
• Deploy immediately after merging

Trunk-Based Development

• Small, frequent commits directly to main or short-lived branches
• Feature flags hide incomplete work
• Continuous integration runs on every commit
• Releases are snapshots of main at any time

Stashing Changes and Applying Patches

Stash temporarily saves uncommitted changes when you need to switch contexts. It's like a clipboard for your working directory.

# Save current changes to stash
git stash

# Save with a descriptive message
git stash save "WIP: login feature"

# List all stashes
git stash list

# Apply most recent stash (keep in stash)
git stash apply

# Apply and remove from stash
git stash pop

# Apply specific stash
git stash apply stash@{2}

# Create branch from stash
git stash branch new-branch

# Clear all stashes
git stash clear

Key Points

• Stashes are stack-based (last in, first out)
• Untracked files aren't stashed by default (use u)
• Stashes are local only, never pushed
• Use for quick context switches, not long-term storage

Rewriting History

Sometimes you need to clean up commits before sharing them. Amend and interactive rebase let you modify local history, but never rewrite public commits.

# Amend last commit (change message or add files)
git commit --amend -m "Better commit message"

# Add forgotten files to last commit
git add forgotten-file.txt
git commit --amend --no-edit

# Interactive rebase last 3 commits
git rebase -i HEAD~3

# Rebase from specific commit
git rebase -i abc123

# Squash multiple commits into one (during rebase)
# In rebase editor: change 'pick' to 'squash'

Rebase Interactive Options

• pick - use the commit
• reword - change commit message
• edit - modify commit content
• squash - combine with previous commit
• fixup - like squash but discard message
• drop - remove commit

Git LFS for Large Files

Git LFS (Large File Storage) replaces large files with text pointers while storing the actual content on a remote server. This keeps repositories fast and clones small.

Key Points

• Install LFS separately: git lfs install
• Track file types: git lfs track "*.psd"
• LFS files are downloaded only when needed
• Requires server support (GitHub, GitLab support LFS)
• Great for binaries, assets, datasets, and media

# Install LFS in repository
git lfs install

# Track file patterns
git lfs track "*.zip"
git lfs track "assets/**"

# View tracked patterns
git lfs track

# List LFS files in repository
git lfs ls-files

# Migrate existing files to LFS
git lfs migrate import --include="*.psd" --everything

Security in Git and GitHub

Secure your code and access with proper authentication and repository protection. Never commit secrets, tokens, or passwords.

# Generate SSH key
ssh-keygen -t ed25519 -C "your.email@example.com"

# Add key to ssh-agent
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

# View public key to add to GitHub
cat ~/.ssh/id_ed25519.pub

# Test SSH connection
ssh -T git@github.com

GitHub Security Features

• Branch protection rules prevent direct pushes to main
• Require pull request reviews before merging
• Require status checks to pass
• Dependabot alerts for vulnerable dependencies
• Secret scanning detects committed credentials
• Personal access tokens replace passwords for HTTPS 

Workflow Summary

The complete Git workflow connects local changes through staging, commits, and branches to remote collaboration and deployment.

# Complete feature workflow example
git checkout -b feature/new-dashboard
git add .
git commit -m "Add dashboard components"
git push -u origin feature/new-dashboard
# Open pull request on GitHub
# Address review comments
git add .
git commit -m "Fix review feedback"
git push
# Merge when approved
git checkout main
git pull origin main
git branch -d feature/new-dashboard

Conclusion

Git and GitHub form the backbone of modern software development, enabling teams to collaborate efficiently while maintaining complete project history. This cheat sheet condenses years of best practices into a quick-reference format designed for daily use. Master these commands and workflows to work confidently whether you're solo coding or contributing to large open-source projects. Bookmark this guide, share it with your teammates, and return whenever you need to refresh your Git knowledge. If you found this useful, pass it along to fellow developers.

Linux in One Shot – The Ultimate Cheatsheet is a compact reference guide that brings together the most important Linux concepts, commands, and tools in one place. It is designed for developers, system administrators, and students who want a quick yet practical overview of Linux without going through lengthy documentation. From basic terminal commands and file management to permissions, networking, and process control, this cheatsheet provides concise explanations and organized command tables for fast learning and easy reference.

Introduction to Linux

Linux is an open-source Unix-like operating system kernel first released by Linus Torvalds in 1991. It powers everything from servers and embedded systems to desktops and mobile devices. The operating system is distributed through various packages called distributions, with popular ones including Ubuntu, CentOS, Debian, and Fedora. Understanding Linux commands and concepts is essential for developers and system administrators working with servers, cloud infrastructure, or development environments.

Linux File System Structure

The Linux file system follows the Filesystem Hierarchy Standard (FHS), organizing all files and directories under the root directory ( / ). Everything in Linux is treated as a file, including hardware devices and system processes. Understanding this hierarchy helps you navigate and manage the system effectively.

Essential Linux Commands

These fundamental commands form the foundation of working with Linux. Mastering them enables you to navigate the system, get help, and perform basic operations efficiently.

# Get help with any command
man ls
ls --help

# Check system information
uname -a
hostname

File and Directory Management

Navigating and manipulating files and directories are daily tasks in Linux. These commands allow you to create, move, copy, delete, and search for files efficiently.

# Create nested directories
mkdir -p projects/website/css

# Copy with verbose output
cp -v index.html backup/

# Find files modified in last 7 days
find /home -type f -mtime -7

File Permissions and Ownership

Linux uses a robust permission system to control access to files and directories. Each file has three permission sets (owner, group, others) with read, write, and execute capabilities. Understanding permissions is crucial for system security and multi-user environments.

Permission values:

• r (read) = 4
• w (write) = 2
• x (execute) = 1

# View file permissions
ls -l filename

# Make script executable for owner
chmod u+x script.sh

# Set specific permissions using octal notation
chmod 644 document.txt
chmod 755 directory/

# Change ownership recursively
chown -R www-data:www-data /var/www

Process Management

Processes are running instances of programs on your Linux system. Managing them effectively helps maintain system performance and troubleshoot issues. Linux provides commands to view, control, and terminate processes.

# Find process by name
ps aux | grep nginx

# Kill all processes matching pattern
pkill -f "python app.py"

# Run process in background
long_running_task &

# View process tree
pstree -p

Networking Commands

Networking commands help you configure, monitor, and troubleshoot network connections. These tools are essential for server administration, debugging connectivity issues, and ensuring services are accessible.

# Check if service is listening on port
ss -tulpn | grep :80

# Download entire website
wget -r -l 3 <https://example.com>

# Test HTTP headers
curl -I <https://api.github.com>

Package Management

Package managers simplify software installation, updates, and removal. Different Linux distributions use different package managers, but the concepts remain similar. Understanding package management helps maintain system integrity and track installed software.

Debian/Ubuntu (apt):

Red Hat/CentOS (yum/dnf):

# Debian/Ubuntu examples
sudo apt update && sudo apt upgrade -y
sudo apt install nginx
apt search "web server"

# Red Hat examples
sudo yum install httpd
sudo yum list installed | grep mysql

# Check installed package version
dpkg -l | grep python3
rpm -qa | grep kernel

Disk and Storage Commands

Managing disk space, partitions, and storage devices is critical for system maintenance. These commands help you monitor usage, mount devices, and manage file systems effectively.

# Check disk usage in human-readable format
df -h

# Find largest files in directory
du -ah /var | sort -rh | head -10

# Create disk image
dd if=/dev/sda of=/backup/disk.img bs=4M

# Check disk I/O performance
iostat -x 1

System Monitoring

Proactive system monitoring helps identify issues before they become critical. These commands provide real-time and historical data about system resources, user activity, and overall system health.

# Monitor memory usage in real-time
watch -n 2 free -m

# Check systemd service logs
journalctl -u nginx.service --since "5 minutes ago"

# View last 50 kernel messages
dmesg | tail -50

# Monitor CPU temperature
watch -n 1 sensors

Bash Scripting Basics

Shell scripting automates repetitive tasks and combines multiple commands into reusable programs. Bash is the default shell on most Linux systems and provides powerful programming constructs for system administration.

#!/bin/bash
# Simple backup script

backup_dir="/backup/$(date +%Y%m%d)"
source_dir="/home/user/documents"

# Create backup directory
mkdir -p "$backup_dir"

# Copy and compress files
tar -czf "$backup_dir/backup.tar.gz" "$source_dir"

# Check if successful
if [ $? -eq 0 ]; then
    echo "Backup completed: $backup_dir"
    logger "Backup successful: $backup_dir"
else
    echo "Backup failed" >&2
    exit 1
fi

Useful Linux Tips

These practical tips and shortcuts improve productivity and help you work more efficiently with the Linux command line. They're gathered from years of real-world usage.

Command-line shortcuts:

• Tab: Auto-complete commands and filenames
• Ctrl + C: Terminate current command
• Ctrl + Z: Suspend current process
• Ctrl + D: Exit current shell
• Ctrl + R: Search command history
• !!: Repeat last command
• !$: Use last argument from previous command
• Ctrl + A/E: Move to beginning/end of line

Redirection operators:

command > file    # Redirect stdout to file
command >> file   # Append stdout to file
command 2> file   # Redirect stderr to file
command &> file   # Redirect both stdout and stderr
command1 | command2   # Pipe output to another command

Quick system checks:

# Check Linux distribution
cat /etc/os-release

# Check kernel version
uname -r

# List available shells
cat /etc/shells

# Show environment variables
env | grep PATH

# Find command location
which python3
whereis nginx

Conclusion

Linux is a powerful operating system used widely in development, servers, and cloud environments. This cheatsheet brings together essential Linux commands and concepts in one place so you can quickly reference them while learning or working in the terminal. Regular practice is the best way to become comfortable with Linux and improve your command-line skills.

If you found this cheatsheet helpful, consider sharing it with your friends or colleagues who are learning Linux.

A complete computer networks cheat sheet that simplifies core networking concepts including OSI layers, TCP/IP model, network topologies, IP addressing, subnetting, networking protocols, devices, and security basics. Perfect for quick revision and interview preparation.

Introduction to Computer Networks

What is a Computer Network

A computer network is a collection of interconnected devices that can communicate and share resources with each other. These devices, ranging from computers and servers to printers and smartphones, use transmission media and protocols to exchange data.

Why Networks are Needed

• Resource sharing (printers, storage, applications)
• Cost reduction through hardware and software sharing
• Improved communication and collaboration
• Centralized data management and backup
• Increased reliability through alternative paths

Types of Networks

Network Topologies

Bus Topology: All devices connect to a single central cable. Simple but a break in the main cable brings down the entire network.

Star Topology: All devices connect to a central hub or switch. Easy to install and manage, but central device failure affects all connected devices.

Ring Topology: Each device connects to two other devices, forming a circular path. Data travels in one direction, reducing collision risk.

Mesh Topology: Every device connects to every other device. Provides high redundancy and reliability but expensive to implement.

Hybrid Topology: Combination of two or more different topologies. Used in larger networks to leverage benefits of multiple topologies.

OSI Model

The Open Systems Interconnection (OSI) model is a conceptual framework that standardizes networking functions into seven distinct layers.

The Seven Layers Explained

Layer 1: Physical Layer Handles the physical connection between devices. Converts data into bits for transmission over physical media like cables or radio waves. Defines voltage levels, cable specifications, and data rates.

Layer 2: Data Link Layer Provides node-to-node data transfer and handles error correction from the physical layer. Organizes bits into frames and manages MAC addresses. Switches and bridges operate at this layer.

Layer 3: Network Layer Responsible for packet forwarding and routing. Determines the best path for data transmission using logical addresses (IP addresses). Routers operate at this layer.

Layer 4: Transport Layer Ensures complete data transfer with error recovery and flow control. Segments and reassembles data. TCP and UDP protocols work here.

Layer 5: Session Layer Manages sessions between applications. Establishes, maintains, and terminates connections. Handles authentication and reconnection after interruptions.

Layer 6: Presentation Layer Translates data between application and network formats. Handles encryption, compression, and data formatting. Ensures data from sender is readable by receiver.

Layer 7: Application Layer Closest to end users. Provides network services directly to applications. HTTP, FTP, SMTP protocols operate at this layer.

OSI Layers and Functions

Mnemonic to Remember the Layers

Please Do Not Throw Sausage Pizza Away (Physical, Data Link, Network, Transport, Session, Presentation, Application)

TCP/IP Model

The TCP/IP model is a practical, streamlined framework based on standard protocols used on the internet.

The Four Layers

Application Layer Combines the functions of OSI's Application, Presentation, and Session layers. Handles high-level protocols, data representation, and session management. Includes HTTP, FTP, SMTP, and DNS.

Transport Layer Corresponds to OSI's Transport layer. Provides reliable or unreliable delivery and error checking. Uses TCP for reliable connections and UDP for faster, connectionless communication.

Internet Layer Equivalent to OSI's Network layer. Handles logical addressing and routing. The core protocol is IP, supported by ICMP and ARP.

Network Access Layer Combines OSI's Data Link and Physical layers. Manages hardware addressing and physical transmission over the network medium.

Protocol Examples by Layer

OSI vs TCP/IP Comparison

Networking Devices

Device Descriptions

Hub: Basic device that connects multiple Ethernet devices. Broadcasts data to all ports regardless of destination. Operates at Physical layer.

Switch: Intelligent device that forwards data only to specific destination ports using MAC addresses. Operates at Data Link layer.

Router: Connects different networks and directs data packets between them using IP addresses. Operates at Network layer.

Bridge: Connects two network segments and filters traffic based on MAC addresses. Reduces unnecessary traffic.

Repeater: Amplifies and regenerates signals to extend network distance. Operates at Physical layer.

Gateway: Translates data between different network protocols or architectures. Can operate at multiple layers.

Modem: Modulates and demodulates signals for transmission over telephone or cable lines. Converts digital to analog and vice versa.

Access Point: Allows wireless devices to connect to a wired network. Extends wireless coverage.

Device Reference Table

IP Addressing

IPv4 Format

IPv4 addresses are 32-bit numbers written as four decimal octets separated by dots. Each octet ranges from 0 to 255.

Example: 192.168.1.10

IPv6 Basics

IPv6 addresses are 128-bit hexadecimal numbers written as eight groups of four hexadecimal digits separated by colons. Developed to address IPv4 exhaustion.

Example: 2001:0db8:85a3:0000:0000:8a2e:0370:7334

Public vs Private IP Addresses

Static vs Dynamic IP

Static IP: Manually assigned, permanent address. Used for servers and devices needing constant access.

Dynamic IP: Automatically assigned by DHCP server. Changes periodically. Common for client devices.

Loopback Address

127.0.0.1 (IPv4) or ::1 (IPv6) used to test network software on local machine without sending packets over network.

Subnet Mask

A 32-bit number that divides an IP address into network and host portions. Uses binary 1s for network bits and 0s for host bits.

Example: 255.255.255.0 indicates first 24 bits are network portion.

CIDR Notation

Classless Inter-Domain Routing notation specifies IP address and network prefix length.

Example: 192.168.1.0/24 indicates 24 bits for network portion, 8 bits for hosts.

Subnetting

Why Subnetting is Needed

• Efficient IP address utilization
• Improved network performance through reduced broadcast traffic
• Enhanced security by isolating network segments
• Simplified network management

Basic Formula

Number of subnets = 2^n where n is number of bits borrowed from host portion Number of hosts per subnet = 2^h - 2 where h is remaining host bits

Simple Example Calculation

Network: 192.168.1.0/24 Requirement: Create 4 subnets

Borrow 2 bits from host portion (2^2 = 4 subnets) New subnet mask: /26 or 255.255.255.192

Resulting subnets:

• 192.168.1.0/26 (hosts: 192.168.1.1 - 192.168.1.62)
• 192.168.1.64/26 (hosts: 192.168.1.65 - 192.168.1.126)
• 192.168.1.128/26 (hosts: 192.168.1.129 - 192.168.1.190)
• 192.168.1.192/26 (hosts: 192.168.1.193 - 192.168.1.254)

Host Calculation

For /26 subnet (64 addresses total):

• Network address: first address
• Broadcast address: last address
• Usable hosts: 62 (total - 2)

Common Networking Protocols

Network Security Basics

Firewall

A security system that monitors and controls incoming and outgoing network traffic based on predetermined security rules. Can be hardware-based, software-based, or both. Creates a barrier between trusted internal and untrusted external networks.

VPN

Virtual Private Network creates encrypted connections over public networks. Provides secure remote access to private networks, masking user location and encrypting all transmitted data.

SSL/TLS

Secure Socket Layer and Transport Layer Security are cryptographic protocols providing secure communication over networks. Used extensively in web browsing (HTTPS), email, and VoIP. TLS is the modern successor to SSL.

Encryption

Process of converting data into unreadable format using algorithms and keys. Ensures data confidentiality during transmission and storage. Common types include symmetric (same key for encryption/decryption) and asymmetric (public/private key pairs).

Authentication

Verification of user or device identity before granting network access. Methods include passwords, biometrics, digital certificates, and multi-factor authentication combining multiple verification methods.

Important Networking Commands

Conclusion

This cheat sheet explains the basic networking concepts like OSI & TCP/IP models, IP addressing, subnetting, common protocols, and basic security. It also includes useful command-line tools for network troubleshooting.

It’s helpful for quick learning, interview preparation, and revision of networking fundamentals.

If you like it, please share it with your friends!

Master Spring Boot from zero to hero with this comprehensive, no-fluff reference. Whether you're just starting out, preparing for interviews, or building a project, this cheat sheet packs everything you need — annotations, config, security, JPA, testing, deployment — with practical examples you can use immediately. No prior Spring experience required!

1. What is Spring Boot?

Spring Boot is an extension of the Spring Framework that eliminates boilerplate configuration and setup. It provides embedded servers, auto-configuration, and production-ready features so developers can focus entirely on business logic rather than infrastructure.

Key Benefits:

• Zero XML configuration
• Embedded Tomcat, Jetty, or Undertow
• Auto-configures beans based on classpath
• Production monitoring via Actuator
• Single executable JAR deployment

Spring Boot vs Spring Framework:

2. Project Structure

A standard Spring Boot project follows a layered package structure. Each folder represents a layer with a single responsibility, making the codebase easy to navigate and maintain.

src/main/java/com/example/
├── controller/       ← HTTP request/response handling
├── service/          ← Business logic
├── repository/       ← Database access
├── model/            ← Entities and DTOs
└── Application.java  ← Main entry point

src/main/resources/
├── application.properties  ← App configuration
└── static/                 ← Static files

3. Main Application Class

@SpringBootApplication is a convenience annotation that combines three annotations into one. When SpringApplication.run() is called, it creates the ApplicationContext, triggers auto-configuration, and starts the embedded server.

@SpringBootApplication
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}

@SpringBootApplication internally combines:

• @Configuration — marks class as bean source
• @EnableAutoConfiguration — enables auto-config
• @ComponentScan — scans current package and sub-packages

4. Spring Starters

Starters are pre-packaged dependency bundles for specific features. Instead of manually finding and matching compatible library versions, you add one starter and Spring Boot handles the rest.

5. Core Spring Concepts

Inversion of Control (IoC)

IoC means Spring takes control of object creation instead of you using the new keyword manually. The Spring IoC container creates, wires, and manages all objects (beans) throughout the application lifecycle.

// Without IoC — you manage everything
UserService service = new UserService(new UserRepository());

// With IoC — Spring manages everything
@Autowired
UserService service;

Dependency Injection (DI)

DI is the mechanism through which IoC is implemented. Spring injects required dependencies into a class automatically, removing tight coupling between components.

Three types of DI:

// 1. Constructor Injection — Recommended
@Service
public class OrderService {
    private final PaymentService paymentService;

    public OrderService(PaymentService paymentService) {
        this.paymentService = paymentService;
    }
}

// 2. Setter Injection — for optional dependencies
@Service
public class NotificationService {
    private EmailService emailService;

    @Autowired
    public void setEmailService(EmailService emailService) {
        this.emailService = emailService;
    }
}

// 3. Field Injection — avoid in production
@Autowired
private UserRepository userRepository;

Beans

A Bean is any Java object created and managed by the Spring IoC container. You register beans using stereotype annotations or @Bean inside a @Configuration class.

// Via annotation
@Component
public class EmailValidator { }

// Via @Configuration
@Configuration
public class AppConfig {
    @Bean
    public PasswordEncoder passwordEncoder() {
        return new BCryptPasswordEncoder();
    }
}

Stereotype Annotations

6. Configuration

Spring Boot externalizes configuration so the same JAR can run in dev, test, and production without code changes. Configuration sources follow a priority order — higher sources override lower ones.

Priority order (highest to lowest):

1. Command line arguments
2. Java system properties
3. Environment variables
4. application.properties or application.yml
5. Default properties

application.properties:

server.port=8080
spring.datasource.url=jdbc:mysql://localhost:3306/mydb
spring.datasource.username=root
spring.datasource.password=${DB_PASSWORD}
spring.jpa.hibernate.ddl-auto=update
spring.jpa.show-sql=true
logging.level.com.example=DEBUG

application.yml (same config, hierarchical format):

server:
  port: 8080
spring:
  datasource:
    url: jdbc:mysql://localhost:3306/mydb
    username: root
    password: ${DB_PASSWORD}
  jpa:
    hibernate:
      ddl-auto: update
    show-sql: true

Profiles

Profiles let you define environment-specific configuration files. When a profile is active, Spring Boot loads both application.properties and the profile-specific file, with the profile file taking priority.

# Activate in application.properties
spring.profiles.active=dev

# Or at runtime
java -jar app.jar --spring.profiles.active=prod

Create separate files per environment:

• application-dev.properties
• application-prod.properties
• application-test.properties

7. REST API Development

Spring Boot makes building REST APIs simple using annotation-based controllers and automatic JSON serialization via Jackson. REST follows stateless communication where each request contains all the information needed to process it.

HTTP Method to Annotation mapping:

Complete CRUD Controller:

@RestController
@RequestMapping("/api/users")
public class UserController {

    private final UserService userService;

    public UserController(UserService userService) {
        this.userService = userService;
    }

    @GetMapping
    public List<User> getAll() {
        return userService.getAll();
    }

    @GetMapping("/{id}")
    public ResponseEntity<User> getById(@PathVariable Long id) {
        return ResponseEntity.ok(userService.getById(id));
    }

    @PostMapping
    public ResponseEntity<User> create(@Valid @RequestBody UserRequest request) {
        return ResponseEntity.status(201).body(userService.create(request));
    }

    @PutMapping("/{id}")
    public ResponseEntity<User> update(@PathVariable Long id,
                                       @Valid @RequestBody UserRequest request) {
        return ResponseEntity.ok(userService.update(id, request));
    }

    @DeleteMapping("/{id}")
    public ResponseEntity<Void> delete(@PathVariable Long id) {
        userService.delete(id);
        return ResponseEntity.noContent().build();
    }
}

Request data extraction:

// From URL path: /users/5
@GetMapping("/{id}")
public User getUser(@PathVariable Long id) { }

// From query string: /users?role=admin&page=0
@GetMapping
public List<User> getUsers(
    @RequestParam String role,
    @RequestParam(defaultValue = "0") int page) { }

// From request body (JSON → Java object)
@PostMapping
public User create(@RequestBody UserRequest request) { }

HTTP Status Codes:

8. Validation and Exception Handling

Bean Validation API provides declarative constraints on request objects. When @Valid is used in a controller, Spring automatically validates the incoming data before the method executes and throws MethodArgumentNotValidException if it fails.

Validation annotations:

DTO with validation:

public class UserRequest {

    @NotBlank(message = "Name is required")
    @Size(min = 2, max = 50)
    private String name;

    @NotBlank
    @Email(message = "Invalid email format")
    private String email;

    @Min(value = 18, message = "Must be at least 18")
    private Integer age;
}

Global Exception Handler:

@RestControllerAdvice centralizes exception handling across all controllers. Without it, Spring returns ugly stack traces. With it, every error returns a clean, consistent JSON response.

@RestControllerAdvice
public class GlobalExceptionHandler {

    @ExceptionHandler(ResourceNotFoundException.class)
    public ResponseEntity<ErrorResponse> handleNotFound(ResourceNotFoundException ex) {
        return ResponseEntity.status(404)
                .body(new ErrorResponse(404, ex.getMessage()));
    }

    @ExceptionHandler(MethodArgumentNotValidException.class)
    public ResponseEntity<Map<String, String>> handleValidation(
            MethodArgumentNotValidException ex) {
        Map<String, String> errors = new HashMap<>();
        ex.getBindingResult().getFieldErrors()
          .forEach(e -> errors.put(e.getField(), e.getDefaultMessage()));
        return ResponseEntity.badRequest().body(errors);
    }

    @ExceptionHandler(Exception.class)
    public ResponseEntity<ErrorResponse> handleGeneric(Exception ex) {
        return ResponseEntity.status(500)
                .body(new ErrorResponse(500, "Unexpected error occurred"));
    }
}

9. Database Integration — Spring Data JPA

Spring Data JPA sits on top of JPA (specification) and Hibernate (implementation) to eliminate manual SQL and ResultSet handling. You define an interface, and Spring generates the full implementation at runtime.

Entity Mapping

@Entity
@Table(name = "users")
public class User {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(nullable = false)
    private String name;

    @Column(unique = true, nullable = false)
    private String email;

    @Column(name = "created_at")
    private LocalDateTime createdAt;
}

Primary Key Generation Strategies:

Repository Layer

@Repository
public interface UserRepository extends JpaRepository<User, Long> {

    // Derived query — Spring generates SQL from method name
    Optional<User> findByEmail(String email);
    List<User> findByNameContaining(String keyword);
    List<User> findByAgeGreaterThanEqual(int age);

    // Custom JPQL query
    @Query("SELECT u FROM User u WHERE u.age > :age ORDER BY u.name ASC")
    List<User> findUsersOlderThan(@Param("age") int age);

    // Native SQL query
    @Query(value = "SELECT * FROM users WHERE email LIKE %:domain%",
           nativeQuery = true)
    List<User> findByEmailDomain(@Param("domain") String domain);

    // Pagination
    Page<User> findByDepartment(String department, Pageable pageable);
}

Pagination and Sorting:

// Page 0, 10 items per page, sorted by name ascending
Pageable pageable = PageRequest.of(0, 10, Sort.by("name").ascending());
Page<User> page = userRepository.findAll(pageable);

page.getContent();       // list of users
page.getTotalPages();    // total pages
page.getTotalElements(); // total count

Entity Relationships

One-to-One:

@Entity
public class User {
    @OneToOne(cascade = CascadeType.ALL)
    @JoinColumn(name = "profile_id")
    private Profile profile;
}

One-to-Many / Many-to-One:

@Entity
public class User {
    @OneToMany(mappedBy = "user", cascade = CascadeType.ALL)
    private List<Order> orders = new ArrayList<>();
}

@Entity
public class Order {
    @ManyToOne
    @JoinColumn(name = "user_id")
    private User user;
}

Many-to-Many:

@Entity
public class Student {
    @ManyToMany(cascade = CascadeType.ALL)
    @JoinTable(
        name = "student_course",
        joinColumns = @JoinColumn(name = "student_id"),
        inverseJoinColumns = @JoinColumn(name = "course_id")
    )
    private Set<Course> courses = new HashSet<>();
}

Fetch Types:

Prefer LAZY for collections to avoid loading unnecessary data.

Avoid infinite recursion in bidirectional relationships:

@OneToMany(mappedBy = "user")
@JsonManagedReference
private List<Order> orders;

@ManyToOne
@JsonBackReference
private User user;

10. Transaction Management

Transactions ensure that a group of database operations either all succeed or all fail together, maintaining data integrity. Spring Boot provides declarative transaction management through the @Transactional annotation.

@Service
@Transactional
public class OrderService {

    @Transactional(readOnly = true)   // Optimizes read-only operations
    public Order getOrder(Long id) {
        return orderRepository.findById(id).orElseThrow();
    }

    @Transactional(rollbackFor = Exception.class)
    public Order createOrder(OrderRequest request) {
        Order order = orderRepository.save(buildOrder(request));
        inventoryService.deductStock(request.getItems());
        return order;
    }
}

Propagation Types:

Isolation Levels:

11. Async Processing

@Async allows methods to run in a separate thread so the caller doesn't wait for completion. It is useful for sending emails, notifications, or any long-running task that doesn't need to block the main request.

// Enable async in config
@Configuration
@EnableAsync
public class AsyncConfig {

    @Bean(name = "taskExecutor")
    public Executor taskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(5);
        executor.setMaxPoolSize(10);
        executor.setQueueCapacity(100);
        executor.setThreadNamePrefix("Async-");
        executor.initialize();
        return executor;
    }
}

// Use in service
@Service
public class EmailService {

    @Async("taskExecutor")
    public CompletableFuture<String> sendEmail(String to, String content) {
        // runs in separate thread
        return CompletableFuture.completedFuture("Sent");
    }
}

12. Spring Security

Spring Security intercepts every incoming request through a filter chain before it reaches your controllers. It handles authentication (who are you?) and authorization (what can you do?) in a structured, extensible way.

Security Filter Chain

@Configuration
@EnableWebSecurity
public class SecurityConfig {

    @Bean
    public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
        http
            .csrf(csrf -> csrf.disable())
            .sessionManagement(s ->
                s.sessionCreationPolicy(SessionCreationPolicy.STATELESS))
            .authorizeHttpRequests(auth -> auth
                .requestMatchers("/api/auth/**").permitAll()
                .requestMatchers("/api/admin/**").hasRole("ADMIN")
                .anyRequest().authenticated()
            )
            .authenticationProvider(authenticationProvider())
            .addFilterBefore(jwtFilter, UsernamePasswordAuthenticationFilter.class);

        return http.build();
    }

    @Bean
    public PasswordEncoder passwordEncoder() {
        return new BCryptPasswordEncoder();
    }

    @Bean
    public DaoAuthenticationProvider authenticationProvider() {
        DaoAuthenticationProvider provider = new DaoAuthenticationProvider();
        provider.setUserDetailsService(userDetailsService);
        provider.setPasswordEncoder(passwordEncoder());
        return provider;
    }
}

13. JWT Authentication

JWT (JSON Web Token) is a stateless authentication mechanism where the server issues a signed token after login. The client stores the token and sends it with every request — no session storage needed on the server.

JWT Structure:

eyJhbGciOiJIUzI1NiJ9  ← Header (algorithm)
.eyJzdWIiOiJqb2huIn0  ← Payload (user data, expiry)
.SflKxwRJSMeKKF2QT4f  ← Signature (tamper-proof)

Never store passwords or secrets in the payload — it is Base64 encoded, not encrypted.

@Service
public class JwtService {

    @Value("${jwt.secret}")
    private String secret;

    private Key getSigningKey() {
        return Keys.hmacShaKeyFor(secret.getBytes());
    }

    public String generateToken(String username) {
        return Jwts.builder()
                .setSubject(username)
                .setIssuedAt(new Date())
                .setExpiration(new Date(System.currentTimeMillis() + 86400000))
                .signWith(getSigningKey(), SignatureAlgorithm.HS256)
                .compact();
    }

    public String extractUsername(String token) {
        return Jwts.parserBuilder()
                .setSigningKey(getSigningKey()).build()
                .parseClaimsJws(token).getBody().getSubject();
    }

    public boolean isTokenValid(String token, String username) {
        return username.equals(extractUsername(token)) && !isExpired(token);
    }
}

JWT Filter:

@Component
@RequiredArgsConstructor
public class JwtAuthenticationFilter extends OncePerRequestFilter {

    private final JwtService jwtService;
    private final UserDetailsService userDetailsService;

    @Override
    protected void doFilterInternal(HttpServletRequest request,
                                    HttpServletResponse response,
                                    FilterChain chain)
                                    throws ServletException, IOException {

        String authHeader = request.getHeader("Authorization");
        if (authHeader == null || !authHeader.startsWith("Bearer ")) {
            chain.doFilter(request, response);
            return;
        }

        String token = authHeader.substring(7);
        String username = jwtService.extractUsername(token);

        if (username != null &&
            SecurityContextHolder.getContext().getAuthentication() == null) {

            UserDetails userDetails = userDetailsService.loadUserByUsername(username);

            if (jwtService.isTokenValid(token, userDetails.getUsername())) {
                UsernamePasswordAuthenticationToken authToken =
                    new UsernamePasswordAuthenticationToken(
                        userDetails, null, userDetails.getAuthorities());
                authToken.setDetails(
                    new WebAuthenticationDetailsSource().buildDetails(request));
                SecurityContextHolder.getContext().setAuthentication(authToken);
            }
        }
        chain.doFilter(request, response);
    }
}

14. CORS Configuration

CORS is a browser security restriction that blocks frontend apps on a different origin from calling your backend. The server must explicitly tell the browser which origins, methods, and headers are allowed.

@Bean
public CorsConfigurationSource corsConfigurationSource() {
    CorsConfiguration config = new CorsConfiguration();
    config.setAllowedOrigins(List.of("<http://localhost:3000>"));
    config.setAllowedMethods(List.of("GET","POST","PUT","DELETE","OPTIONS"));
    config.setAllowedHeaders(List.of("*"));
    config.setAllowCredentials(true);

    UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
    source.registerCorsConfiguration("/**", config);
    return source;
}

// Wire it in SecurityFilterChain
http.cors(cors -> cors.configurationSource(corsConfigurationSource()))

15. Logging with @Slf4j

Spring Boot uses SLF4J with Logback by default. The @Slf4j annotation from Lombok auto-generates a logger instance so you can start logging immediately without boilerplate.

Log Levels (lowest to highest priority):

@Slf4j
@Service
public class PaymentService {

    public PaymentResult process(PaymentRequest request) {
        log.info("Processing payment for order: {}", request.getOrderId());
        log.debug("Payment amount: {}", request.getAmount());

        try {
            PaymentResult result = gateway.charge(request);
            log.info("Payment successful, transaction: {}", result.getTransactionId());
            return result;
        } catch (PaymentGatewayException e) {
            log.error("Payment failed for order {}: {}", request.getOrderId(), e.getMessage(), e);
            throw e;
        }
    }
}

Configuration:

logging.level.root=INFO
logging.level.com.example=DEBUG
logging.level.org.hibernate.SQL=DEBUG
logging.file.name=logs/application.log
logging.pattern.console=%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n

Best practices:

• Always use parameterized logging log.info("User: {}", name) not string concatenation
• Never log passwords, tokens, or card numbers
• Always include the exception object in log.error() for stack trace
• Use DEBUG and TRACE only — never INFO — for internal flow details

16. Testing

Spring Boot provides excellent testing support out of the box. Following the testing pyramid — many unit tests, some integration tests, few end-to-end tests — gives you fast feedback with reliable coverage.

Unit Testing with Mockito

Unit tests verify a single class in isolation. Mock all dependencies so the test only exercises the class under test.

@ExtendWith(MockitoExtension.class)
class UserServiceTest {

    @Mock
    private UserRepository userRepository;

    @Mock
    private PasswordEncoder passwordEncoder;

    @InjectMocks
    private UserService userService;

    @Test
    void shouldCreateUserSuccessfully() {
        // Arrange
        UserRequest request = new UserRequest("john", "pass", "john@example.com");
        User saved = new User(1L, "john", "encodedPass", "john@example.com");

        when(passwordEncoder.encode("pass")).thenReturn("encodedPass");
        when(userRepository.save(any(User.class))).thenReturn(saved);

        // Act
        User result = userService.createUser(request);

        // Assert
        assertThat(result.getId()).isEqualTo(1L);
        verify(userRepository, times(1)).save(any(User.class));
    }

    @Test
    void shouldThrowWhenUserNotFound() {
        when(userRepository.findById(99L)).thenReturn(Optional.empty());

        assertThatThrownBy(() -> userService.getById(99L))
            .isInstanceOf(ResourceNotFoundException.class);
    }
}

Controller Testing with MockMvc

@WebMvcTest(UserController.class)
class UserControllerTest {

    @Autowired
    private MockMvc mockMvc;

    @MockBean
    private UserService userService;

    @Test
    void shouldReturnUser() throws Exception {
        User user = new User(1L, "john", "john@example.com");
        when(userService.getById(1L)).thenReturn(user);

        mockMvc.perform(get("/api/users/1"))
               .andExpect(status().isOk())
               .andExpect(jsonPath("$.name").value("john"));
    }
}

Repository Testing with @DataJpaTest

@DataJpaTest
class UserRepositoryTest {

    @Autowired
    private UserRepository userRepository;

    @Autowired
    private TestEntityManager entityManager;

    @Test
    void shouldFindByEmail() {
        entityManager.persist(new User(null, "john", "john@example.com"));
        entityManager.flush();

        Optional<User> found = userRepository.findByEmail("john@example.com");
        assertThat(found).isPresent();
    }
}

Testing annotations summary:

17. Spring Boot Actuator

Actuator exposes production-ready endpoints for health checks, metrics, and application info. It is essential for monitoring applications in production environments and integrates easily with tools like Prometheus and Grafana.

management.endpoints.web.exposure.include=health,info,metrics,env,loggers
management.endpoint.health.show-details=always
info.app.name=My Application
info.app.version=1.0.0

Common Endpoints:

18. Spring Boot DevTools

Spring Boot DevTools enhances the development experience by providing automatic restarts, live reload, and development-time optimizations. It should never be used in production — it's strictly a development dependency.

xml

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-devtools</artifactId>
    <scope>runtime</scope>  <!-- Critical: excluded in production -->
    <optional>true</optional>
</dependency>

Key Features

19. Caching

Caching stores the result of expensive operations (database queries, API calls, computations) so repeated requests return instantly without re-executing the underlying logic. Spring Boot integrates with multiple cache providers through a unified abstraction layer.

Enable caching in your main config:

@Configuration
@EnableCaching
public class CacheConfig {
    // Spring Boot auto-configures a ConcurrentMapCache by default
    // For production, swap in Redis or Caffeine (see below)
}

Core caching annotations:

Practical service example:

@Service
public class ProductService {

    // Cache result — skip DB call if key already exists
    @Cacheable(value = "products", key = "#id")
    public Product getById(Long id) {
        return productRepository.findById(id).orElseThrow();
    }

    // Always update cache after save
    @CachePut(value = "products", key = "#result.id")
    public Product update(Long id, ProductRequest request) {
        return productRepository.save(buildProduct(id, request));
    }

    // Remove single entry on delete
    @CacheEvict(value = "products", key = "#id")
    public void delete(Long id) {
        productRepository.deleteById(id);
    }

    // Clear entire cache — useful after bulk operations
    @CacheEvict(value = "products", allEntries = true)
    public void clearAll() { }
}

Cache providers:

Redis setup :

spring.cache.type=redis
spring.data.redis.host=localhost
spring.data.redis.port=6379
spring.cache.redis.time-to-live=600000

Conditional caching:

// Only cache if result is not null
@Cacheable(value = "users", key = "#email", unless = "#result == null")
public User findByEmail(String email) { ... }

// Only cache for admin users
@Cacheable(value = "reports", condition = "#role == 'ADMIN'")
public Report generateReport(String role) { ... }

Best practices:

• Always set a TTL (time-to-live) — unbounded caches grow until OutOfMemoryError
• Use Redis in any multi-instance deployment — in-memory caches are per-JVM and go out of sync
• Cache at the service layer, never the controller or repository layer
• Keep cached objects serializable — required for Redis and most distributed providers
• Use @CacheEvict on every write operation that modifies cached data, or you'll serve stale results

20. API Documentation with Swagger / OpenAPI

Springdoc OpenAPI auto-generates interactive API documentation from your existing controller annotations. It requires zero extra configuration to get started.

<dependency>
    <groupId>org.springdoc</groupId>
    <artifactId>springdoc-openapi-starter-webmvc-ui</artifactId>
    <version>2.3.0</version>
</dependency>

After adding the dependency:

• Swagger UI: http://localhost:8080/swagger-ui.html
• OpenAPI JSON: http://localhost:8080/v3/api-docs

Allow these paths in your security config:

.requestMatchers("/swagger-ui/**", "/v3/api-docs/**").permitAll()

21. Build and Deployment

Spring Boot packages the entire application — including the embedded server — into a single executable JAR. This makes deployment simple and consistent across all environments.

Build commands:

# Maven
mvn clean package
mvn clean package -DskipTests

# Gradle
gradle clean build
gradle clean build -x test

Run the JAR:

java -jar target/app.jar
java -jar target/app.jar --spring.profiles.active=prod
java -jar target/app.jar --server.port=9090
java -Xmx512m -Xms256m -jar target/app.jar

Dockerfile:

FROM openjdk:17-jdk-slim
COPY target/app.jar app.jar
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "/app.jar"]

docker build -t myapp .
docker run -p 8080:8080 -e DB_PASSWORD=secret myapp

Quick Reference — Most Used Annotations

Conclusion

This Spring Boot cheat sheet covers everything from core concepts to production deployment in a concise, practical format. Master these fundamentals and you'll be ready to build production-ready applications with confidence. If you found this helpful, share it with your friends and colleagues — good knowledge grows when shared!

A practical reference for Java backend developers building microservices with Spring Cloud, covering essential concepts such as service discovery, API gateways, centralized configuration, client-side load balancing, resilience patterns, event-driven communication, distributed tracing, and production-ready monitoring and observability.

1. Microservices Fundamentals

Monolith vs Microservices

A monolith packages all application features — UI, business logic, data access — into a single deployable unit. Microservices split that unit into independently deployable services, each owning a single business capability.

Microservices Architecture Principles

• Single Responsibility — each service owns exactly one business domain
• Loose Coupling — services communicate over well-defined APIs; no shared databases
• High Cohesion — related logic lives together inside the same service
• Decentralized Data — each service manages its own datastore
• Design for Failure — assume any downstream call can fail; handle gracefully
• Automate Everything — CI/CD pipelines, containerization, and infrastructure as code

Challenges in Distributed Systems

Distributed systems introduce problems that simply do not exist in a monolith:

• Network latency and partial failures
• Data consistency across service boundaries
• Service discovery — how does Service A find Service B?
• Distributed tracing — how do you follow a request across 10 services?
• Security — each service must authenticate callers
• Operational complexity — many moving parts to monitor and deploy

Service-to-Service Communication

Synchronous communication — the caller waits for a response.

• HTTP/REST using RestTemplate or WebClient
• gRPC for high-performance binary protocol
• OpenFeign for declarative HTTP clients

Asynchronous communication — the caller publishes an event and moves on.

• Apache Kafka for high-throughput event streaming
• RabbitMQ for message queuing with routing
• Event sourcing patterns

When to choose which:

2. Introduction to Spring Cloud

What is Spring Cloud

Spring Cloud is a collection of tools and frameworks built on top of Spring Boot that solves the common infrastructure problems in distributed systems. It is not a single library but an ecosystem of coordinated projects.

Why Spring Cloud

Without Spring Cloud, a team building microservices would need to integrate Eureka, Zuul, Ribbon, Hystrix, and dozens of other libraries manually, write glue code, and maintain compatibility. Spring Cloud provides opinionated, pre-integrated solutions for all of these concerns with sensible defaults.

Spring Cloud Ecosystem

Spring Boot and Spring Cloud Relationship

Spring Boot provides the application runtime, auto-configuration, and embedded server. Spring Cloud builds on Spring Boot's auto-configuration mechanism to wire distributed system infrastructure automatically. Every Spring Cloud service is a Spring Boot application. Spring Cloud adds the coordination layer.

<!-- pom.xml — Spring Cloud BOM manages all version compatibility -->
<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-dependencies</artifactId>
      <version>2023.0.1</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

Always import the Spring Cloud BOM. Never manage Spring Cloud dependency versions individually — version mismatches between modules cause subtle, hard-to-debug failures.

3. Service Discovery

What is Service Discovery

In a static deployment you hard-code IP addresses and ports. In a microservices deployment, services are ephemeral — they start, stop, scale up, and scale down dynamically. Service discovery solves the question: given a service name, what network address should I call right now?

Service Registry

The service registry is the central directory. Every service instance registers itself on startup and sends periodic heartbeats. If a heartbeat stops, the registry removes the instance.

Client-Side vs Server-Side Discovery

Client-Side Discovery — the client queries the registry and picks an instance itself. The client holds the load-balancing logic. Spring Cloud uses this model.

Server-Side Discovery — the client calls a router (load balancer), which queries the registry internally. AWS ALB and Kubernetes Services use this model.

Netflix Eureka

Eureka is the most widely used service registry in the Spring Cloud ecosystem. It is designed for AWS-style availability — it prefers availability over consistency (AP in CAP theorem). When the registry cannot reach enough peers, it enters self-preservation mode and stops evicting instances.

Eureka Server Setup

<!-- dependency -->
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>

@SpringBootApplication
@EnableEurekaServer
public class EurekaServerApplication {
    public static void main(String[] args) {
        SpringApplication.run(EurekaServerApplication.class, args);
    }
}

# application.yml — standalone Eureka server
server:
  port: 8761

eureka:
  instance:
    hostname: localhost
  client:
    register-with-eureka: false   # this server does not register itself
    fetch-registry: false          # this server does not fetch from another registry

Eureka Client Setup

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>

@SpringBootApplication
@EnableDiscoveryClient   // optional from Spring Cloud 2021+, auto-detected
public class OrderServiceApplication {
    public static void main(String[] args) {
        SpringApplication.run(OrderServiceApplication.class, args);
    }
}

# application.yml — Eureka client
spring:
  application:
    name: order-service   # this becomes the service name in the registry

eureka:
  client:
    service-url:
      defaultZone: <http://localhost:8761/eureka/>
  instance:
    prefer-ip-address: true
    lease-renewal-interval-in-seconds: 30
    lease-expiration-duration-in-seconds: 90

Service Registration and Heartbeats

• On startup, the client POSTs its metadata (hostname, IP, port, health URL, status) to the registry
• Every 30 seconds (default), the client sends a heartbeat PUT request
• If the registry receives no heartbeat for 90 seconds, it marks the instance as DOWN
• If the registry loses over 85% of expected heartbeats from all clients, it enters self-preservation mode and stops evicting any instance — this prevents mass deregistration during network partitions

4. API Gateway

API Gateway Pattern

An API Gateway is the single entry point for all client requests. Instead of clients calling each microservice directly, they call the gateway. The gateway handles routing, authentication, rate limiting, SSL termination, request transformation, and logging in one place.

Benefits:

• Clients are decoupled from internal service topology
• Cross-cutting concerns are handled once, not in every service
• Service URLs can change without affecting clients

Spring Cloud Gateway

Spring Cloud Gateway is the modern, reactive replacement for Netflix Zuul. It is built on Project Reactor and Spring WebFlux, which means it is non-blocking and handles high concurrency efficiently.

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>

Routing

A route is the fundamental unit in Spring Cloud Gateway. Each route has:

• An ID for identification
• A URI — the downstream service address
• Predicates — conditions that must match for the route to apply
• Filters — transformations applied to request or response

spring:
  cloud:
    gateway:
      routes:
        - id: order-service-route
          uri: lb://order-service      # lb:// triggers load-balanced discovery
          predicates:
            - Path=/api/orders/**
          filters:
            - StripPrefix=1            # removes /api before forwarding

Predicates

Predicates determine whether a route matches an incoming request. Multiple predicates on one route are ANDed together.

Filters

Filters transform requests or responses. They run in order and fall into two categories: GatewayFilter (per route) and GlobalFilter (all routes).

filters:
  - AddRequestHeader=X-Request-Source, gateway
  - AddResponseHeader=X-Response-Time, 200ms
  - StripPrefix=2
  - RewritePath=/api/(?<segment>.*), /$\\\\{segment}
  - CircuitBreaker=name=orderCircuit,fallbackUri=/fallback
  - RequestRateLimiter=redis-rate-limiter
  - Retry=retries:3,statuses:BAD_GATEWAY

Custom filter with Java:

@Component
public class LoggingGlobalFilter implements GlobalFilter, Ordered {

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        String path = exchange.getRequest().getURI().getPath();
        System.out.println("Incoming request: " + path);
        return chain.filter(exchange)
                    .then(Mono.fromRunnable(() ->
                        System.out.println("Response status: " +
                            exchange.getResponse().getStatusCode())));
    }

    @Override
    public int getOrder() { return -1; }
}

Authentication at Gateway

Centralize JWT validation at the gateway so individual services do not need to repeat it.

@Component
public class JwtAuthenticationFilter implements GlobalFilter {

    @Value("${jwt.secret}")
    private String secret;

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        String token = exchange.getRequest()
                               .getHeaders()
                               .getFirst(HttpHeaders.AUTHORIZATION);

        if (token == null || !token.startsWith("Bearer ")) {
            exchange.getResponse().setStatusCode(HttpStatus.UNAUTHORIZED);
            return exchange.getResponse().setComplete();
        }

        // validate token, then pass claims downstream as header
        exchange.getRequest().mutate()
                .header("X-User-Id", extractUserId(token));
        return chain.filter(exchange);
    }
}

Rate Limiting

Spring Cloud Gateway integrates with Redis for distributed rate limiting using the token bucket algorithm.

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-data-redis-reactive</artifactId>
</dependency>

spring:
  cloud:
    gateway:
      routes:
        - id: rate-limited-route
          uri: lb://product-service
          predicates:
            - Path=/api/products/**
          filters:
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 10    # tokens added per second
                redis-rate-limiter.burstCapacity: 20    # max tokens in bucket
                redis-rate-limiter.requestedTokens: 1   # tokens per request

5. Centralized Configuration

Externalized Configuration

The Twelve-Factor App methodology mandates that configuration is separated from code. In microservices, this means you do not bake environment-specific properties into your JAR. You fetch them at runtime from a central source.

Benefits:

• One change propagates to all service instances without redeployment
• Environment-specific config (dev, staging, prod) managed in one place
• Secrets managed outside source code
• Audit trail via Git history

Spring Cloud Config Server

The Config Server is a standalone Spring Boot application that serves configuration over HTTP. It reads from a Git repository, local file system, HashiCorp Vault, or AWS S3.

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-config-server</artifactId>
</dependency>

@SpringBootApplication
@EnableConfigServer
public class ConfigServerApplication {
    public static void main(String[] args) {
        SpringApplication.run(ConfigServerApplication.class, args);
    }
}

# application.yml — Config Server pointing to Git
server:
  port: 8888

spring:
  cloud:
    config:
      server:
        git:
          uri: <https://github.com/yourorg/config-repo>
          default-label: main
          search-paths: '{application}'    # subfolder per service
          clone-on-start: true

Config Server URL pattern:

/{application}/{profile}[/{label}]

# Examples:
GET <http://localhost:8888/order-service/prod>
GET <http://localhost:8888/order-service/dev/main>

Config Client

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-config</artifactId>
</dependency>

# application.yml — microservice pointing to Config Server
spring:
  application:
    name: order-service         # maps to file name in config repo
  config:
    import: "configserver:<http://localhost:8888>"
  profiles:
    active: dev

File resolution in the Git repo follows this priority order (highest wins):

1. order-service-dev.yml
2. order-service.yml
3. application-dev.yml
4. application.yml

Git-based Configuration Management

Organize the Git repository cleanly:

config-repo/
├── application.yml              # shared across all services
├── application-prod.yml         # shared prod overrides
├── order-service/
│   ├── order-service.yml
│   ├── order-service-dev.yml
│   └── order-service-prod.yml
├── user-service/
│   └── user-service.yml

Refresh Scope

Without @RefreshScope, a bean reads its configuration once at startup. With @RefreshScope, the bean is re-initialized when a /actuator/refresh endpoint is called.

@RestController
@RefreshScope     // re-inject config on refresh
public class FeatureFlagController {

    @Value("${feature.new-checkout-enabled:false}")
    private boolean newCheckoutEnabled;

    @GetMapping("/feature-status")
    public String status() {
        return "New checkout: " + newCheckoutEnabled;
    }
}

Trigger a refresh on one instance:

curl -X POST <http://localhost:8080/actuator/refresh>

Dynamic Configuration Reload with Spring Cloud Bus

To refresh all instances simultaneously without calling each one individually, use Spring Cloud Bus. It broadcasts the refresh event over a message broker (Kafka or RabbitMQ) to every subscribed instance.

<!-- Add to Config Server and all clients -->
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-bus-kafka</artifactId>
</dependency>

# Refresh all instances in one call
curl -X POST <http://localhost:8888/actuator/busrefresh>

6. Client-Side Load Balancing

Load Balancing Concepts

Load balancing distributes incoming requests across multiple instances of a service to avoid overloading any single instance. In the Spring Cloud model, the load balancer runs on the client side — meaning the service making the call decides which instance to call, based on a list it receives from the service registry.

Spring Cloud LoadBalancer

Spring Cloud LoadBalancer replaced Netflix Ribbon as the default client-side load balancer starting from Spring Cloud 2020. It is included automatically when you add the Eureka client dependency.

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-loadbalancer</artifactId>
</dependency>

Using @LoadBalanced with RestTemplate

@Configuration
public class RestTemplateConfig {

    @Bean
    @LoadBalanced   // wraps RestTemplate to resolve service names via registry
    public RestTemplate restTemplate() {
        return new RestTemplate();
    }
}

@Service
public class OrderService {

    @Autowired
    private RestTemplate restTemplate;

    public Product getProduct(Long id) {
        // uses service name, not hardcoded URL
        return restTemplate.getForObject(
            "<http://product-service/api/products/>" + id,
            Product.class
        );
    }
}

Using @LoadBalanced with WebClient (Reactive)

@Configuration
public class WebClientConfig {

    @Bean
    @LoadBalanced
    public WebClient.Builder webClientBuilder() {
        return WebClient.builder();
    }
}

@Service
public class ReactiveOrderService {

    @Autowired
    private WebClient.Builder webClientBuilder;

    public Mono<Product> getProduct(Long id) {
        return webClientBuilder.build()
            .get()
            .uri("<http://product-service/api/products/>" + id)
            .retrieve()
            .bodyToMono(Product.class);
    }
}

Load Balancing Algorithms

Custom Load Balancer Configuration

public class CustomLoadBalancerConfig {

    @Bean
    ReactorLoadBalancer<ServiceInstance> randomLoadBalancer(
            Environment env,
            LoadBalancerClientFactory loadBalancerClientFactory) {
        String name = env.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
        return new RandomLoadBalancer(
            loadBalancerClientFactory.getLazyProvider(name, ServiceInstanceListSupplier.class),
            name
        );
    }
}

// Apply to a specific client
@LoadBalancerClient(name = "product-service", configuration = CustomLoadBalancerConfig.class)
public class ApplicationConfig { }

Integration with Service Discovery

The ServiceInstanceListSupplier fetches the instance list from Eureka (or Consul or Kubernetes). The load balancer then applies its algorithm to that list. Caching is enabled by default with a 35-second TTL to avoid hammering the registry on every request.

spring:
  cloud:
    loadbalancer:
      ribbon:
        enabled: false    # ensure Ribbon is not on classpath
      cache:
        ttl: 35s
        capacity: 256

7. Fault Tolerance and Resilience

Distributed System Failures

In a distributed system with 10 services, if each has 99.9% uptime, the end-to-end availability of a chain through all 10 is approximately 99%. Without resilience patterns, a slow or failing downstream service can cascade into a full system outage through thread pool exhaustion.

Circuit Breaker Pattern

The circuit breaker sits around an external call. It monitors failures. When the failure rate exceeds a threshold, it opens and stops all calls to that service for a cooldown period. After the cooldown, it enters half-open state and allows a limited number of test calls through.

States:

• CLOSED — normal operation; calls pass through
• OPEN — calls fail immediately without hitting the downstream service
• HALF-OPEN — a limited number of probe calls are allowed to test recovery

Resilience4j

Resilience4j is the recommended library for Spring Cloud fault tolerance. It provides circuit breaker, retry, rate limiter, bulkhead, and time limiter as individual composable decorators.

<dependency>
  <groupId>io.github.resilience4j</groupId>
  <artifactId>resilience4j-spring-boot3</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-aop</artifactId>
</dependency>

Circuit Breaker Configuration

resilience4j:
  circuitbreaker:
    instances:
      product-service:
        sliding-window-type: COUNT_BASED
        sliding-window-size: 10               # evaluate last 10 calls
        failure-rate-threshold: 50            # open if 50%+ fail
        wait-duration-in-open-state: 10s      # stay open for 10 seconds
        permitted-number-of-calls-in-half-open-state: 3
        automatic-transition-from-open-to-half-open-enabled: true

@Service
public class ProductService {

    @CircuitBreaker(name = "product-service", fallbackMethod = "getProductFallback")
    public Product getProduct(Long id) {
        return restTemplate.getForObject("<http://product-service/api/products/>" + id, Product.class);
    }

    public Product getProductFallback(Long id, Exception ex) {
        return new Product(id, "Unavailable", 0.0);   // default response
    }
}

Retry Pattern

resilience4j:
  retry:
    instances:
      product-service:
        max-attempts: 3
        wait-duration: 500ms
        retry-exceptions:
          - java.io.IOException
          - java.util.concurrent.TimeoutException
        ignore-exceptions:
          - com.example.ProductNotFoundException

@Retry(name = "product-service", fallbackMethod = "getProductFallback")
public Product getProduct(Long id) {
    return restTemplate.getForObject("...", Product.class);
}