Design a News Feed / Social Media Feed System
A News Feed system is the backbone of every modern social platform—Facebook, Instagram, Twitter/X, LinkedIn. It decides what content a user sees, in what order, and how fast. From a system design perspective, this is one of the hardest problems because it combines massive scale, low latency, uneven traffic, and complex data relationships.
In this blog, we will design a real-world, production-grade News Feed system, explaining why each architectural decision is taken, not just what the decision is.
Index
- Understanding the News Feed Problem
- Functional Requirements
- Non-Functional Requirements (Traffic & Scale)
- High-Level Architecture
- Feed Generation Strategies
- Core System Components & Design Decisions
- Database Architecture & Scalability
- Data Models
- API Design
- Caching Strategy
- Scalability Strategy (Traffic Justification)
- Real Example: Lifecycle of a Feed Item
- Trade-offs & Consistency
- Summary
1. Understanding the News Feed Problem
A news feed system aggregates content from multiple users and delivers it to another user in a personalized, ordered, and fast manner.
The key challenge is imbalance:
- Writes (post creation) are moderate
- Reads (feed views) are extremely high
- Some users (celebrities) generate disproportionate traffic
This imbalance drives most of our design decisions.
2. Functional Requirements
The system should allow users to:
- Create posts (text, image, video)
- Follow/unfollow users
- View a personalized feed
- Like and comment on posts
- Scroll infinitely or paginate feed
Optional but realistic:
- Reposts/shares
- Sponsored content
- Ranking & personalization
3. Non-Functional Requirements (Traffic & Scale)
This section defines why the architecture must scale.
Example Traffic Assumptions
- 30 million daily active users
- Average user follows 400 users
- Each user opens feed 8 times/day
- Each feed request returns 20 posts
➡️ Feed reads/day = 30M × 8 = 240M
➡️ Peak QPS ≈ 200K–300K requests/sec
Key Non-Functional Goals
- Low latency (<200 ms)
- High availability (99.9%+)
- Horizontal scalability
- Eventual consistency
- Graceful degradation
These numbers justify heavy caching, async processing, and distributed databases.
4. High-Level Architecture
At a high level, the system consists of:
- Client (Mobile/Web)
- Load Balancer
- Feed API Service
- User & Follow Service
- Post Service
- Feed Generation Service
- Message Queue
- Cache Layer
- Databases
- Ranking Service
Why this separation?
Each component scales independently based on its traffic pattern (reads vs writes).
5. Feed Generation Strategies
Fan-out on Write
When a user posts:
- Push the post into all followers’ feeds
Why use it?
- Extremely fast feed reads
- Ideal for read-heavy systems
Problem
- A celebrity with 10M followers creates massive write load
Fan-out on Read
When a user opens feed:
- Fetch recent posts from all followed users
Why use it?
- Cheap writes
- Simple logic
Problem
- Slow reads at scale
- Expensive joins/merging
Hybrid Model (Industry Standard)
Design Decision
- Normal users → fan-out on write
- Celebrity users → fan-out on read
This balances write amplification and read latency, making the system scalable.
6. Core System Components & Design Decisions
Load Balancer
Type: L7 (Nginx, HAProxy, AWS ALB)
Routes traffic to stateless API servers.
Why?
- Enables horizontal scaling
- Handles spikes during peak hours
Feed API Service
- Serves feed requests
- Handles pagination
- Reads from cache first
Why stateless?
- Easy scaling
- Fault tolerance
Message Queue
Options: Kafka, AWS SQS, RabbitMQ
Used when:
- A post is created
- Feed updates must be asynchronous
Why?
- Decouples write path from feed generation
- Absorbs traffic spikes
Ranking Service
Applies:
- Time decay
- Engagement signals
- Personalization rules
Advanced systems use ML models here.
7. Database Architecture & Scalability
This is the most critical section.
User & Follow Graph
Options:
- Cassandra (wide-column)
- DynamoDB
- Neo4j (limited use)
Why distributed DB?
- High read/write
- Needs horizontal scaling
- Simple access patterns
Partitioned by user_id.
Posts Database
Characteristics
- Write-heavy
- Append-only
Options
- Cassandra / DynamoDB
- Sharded MySQL (if relational constraints needed)
Why NoSQL often wins
- Linear horizontal scaling
- High write throughput
Feed Storage
- Stores precomputed feed items
- Partitioned by
user_id
Why separate feed store?
- Avoids expensive runtime joins
- Optimized for reads
8. Data Models (Simplified)
User
User(user_id, name, profile_data)
Follow
Follow(follower_id, followee_id)
Post
Post(post_id, user_id, content, created_at)
FeedItem
FeedItem(user_id, post_id, score, created_at)
These models are denormalized intentionally for performance.
9. API Design
Create Post
POST /posts
Request
{
"content": "Hello world!"
}
Response
{
"post_id": "p123",
"status": "created"
}
Get Feed
GET /feed?cursor=xyz
Response
{
"items": [
{
"post_id": "p123",
"author": "user42",
"content": "Hello world!"
}
],
"next_cursor": "abc"
}
Cursor-based pagination avoids performance issues at scale.
10. Caching Strategy
Cache Type: Redis / Memcached
Cached Data
- User feed
- Recent posts
- Follower lists
Why aggressive caching?
- Feed is read extremely frequently
- Cache hit rate >90% drastically reduces DB load
11. Scalability Strategy (Traffic Justification)
As traffic grows:
- API servers scale horizontally
- Cache cluster scales independently
- DB shards increase
- Queue partitions increase
Example
If QPS doubles:
- Add API nodes
- Increase Redis shards
- Add Kafka partitions
This keeps latency stable under load.
12. Real Example: Lifecycle of a Feed Item
Scenario
User A follows User B.
Step 1: Post Creation
- User B creates a post
- Post stored in Posts DB
- Event sent to queue
Step 2: Feed Generation
- Feed service consumes event
- Pushes post into feeds of User B’s followers (fan-out on write)
Step 3: Cache Update
- Feed cache for User A updated
Step 4: Feed Read
- User A opens app
- Feed API reads from cache
- Falls back to DB if cache miss
This lifecycle ensures fast reads and scalable writes.
13. Trade-offs & Consistency
- Eventual consistency accepted
- Slight delays in feed update are fine
- Availability prioritized over strict consistency
This aligns with real-world social systems.
14. Summary
A scalable News Feed system:
- Is read-heavy
- Uses hybrid feed generation
- Relies on caching
- Scales horizontally at every layer
- Accepts eventual consistency
This design reflects how real social platforms operate at scale.
