Building Scalable Full-Stack Systems: Lessons from the Workbench
Reflections on architecting a microservices-based OCR application with built-in observability, real-time status updates, and horizontal scalability—lessons from building scalable-fullstack.
I've been working on a microservices-based OCR application in my workbench/scalable-fullstack repository. Rather than diving into code specifics, I want to share the architectural ideas and design decisions that make this system genuinely scalable, observable, and resilient.
The System: A Microservices OCR Platform
The application extracts text from PDFs and images, stores per-page results, and enables semantic search. What makes it interesting isn't the OCR itself—it's how the system is architected to handle scale, failures, and real-time updates gracefully.
The architecture consists of independent services: a Next.js frontend, a NestJS API, Node.js workers, Python OCR and embedding services, PostgreSQL with vector search, Redis for job queuing, and MinIO for object storage. Each service can be scaled independently, and the system continues operating even when some services are down.
Architecture Diagram
┌───────────────────────────────────────────────────────────────┐
│ USER │
│ Uploads PDF(s) from browser + sees status updates │
└───────────────▲───────────────────────────────────────────────┘
│
│ HTTP Upload
│
┌───────────────┴───────────────────────────────────────────────┐
│ FRONTEND (Next.js) │
│ - Drag & drop upload │
│ - Shows progress + final result │
│ - Calls API to upload files │
└───────────────▲───────────────────────────────────────────────┘
│
│ Sends file
│
┌───────────────┴───────────────────────────────────────────────┐
│ API (NestJS) │
│ - Streams PDF → MinIO Storage │
│ - Creates a DB Record │
│ - Pushes a Job into the Queue │
│ - Sends status to frontend (SSE) │
└───────────────▲────────────────────────────────────────────────┘
│
│ Adds job to queue
│
╔═══════════════╩════════════════════════════════════════════════╗
║ REDIS QUEUE (BullMQ) ║
║ - Holds jobs ║
║ - Retries failed jobs with backoff ║
║ - Allows cancellation ║
║ - Allows multiple workers to process in parallel ║
╚═══════════════╦════════════════════════════════════════════════╝
│
│ Pulls jobs
▼
┌───────────────────────────────────────────────────────────────┐
│ WORKER POOL │
│ (Node.js / BullMQ Workers) │
│ │
│ Each worker does: │
│ 1. Downloads file from MinIO │
│ 2. Calls OCR service → gets extracted text │
│ 3. Saves text to Postgres │
│ 4. Sends text to Embedding service │
│ 5. Stores embeddings in DB │
│ 6. Updates job status (done / failed / cancelled) │
│ │
│ - Automatic retries │
│ - Multiple workers = scales horizontally │
│ - Can cancel jobs on request │
└────────────────────▲──────────────────────────────────────────┘
│
│ Calls microservices
│
┌─────────────┴─────────────┬───────────────────────────┐
│ │ │
▼ ▼ ▼
┌──────────────┐ ┌─────────────────┐ ┌──────────────────┐
│ OCR Service │ │ Embedding Svc │ │ MinIO Storage │
│ (Python) │ │ (Python) │ │ (PDFs/Images) │
│ Extracts │ │ Turns text → │ │ Secure store for │
│ text from │ │ embeddings │ │ uploaded files │
│ PDF / image │ │ (Vectors) │ │ │
└─────▲────────┘ └──────────▲──────┘ └──────────▲──────┘
│ │ │
└──────────────┬───────────────┴──────────────┬────────────┘
│ │
▼ ▼
┌────────────────────────┐ ┌───────────────────────┐
│ Postgres DB │ │ Webhook / API │
│ - Document status │ │ - Worker notifies API │
│ - OCR text │ │ when updates │
│ - Embeddings │ │ - API notifies client │
└────────────────────────┘ └───────────────────────┘
Observability Built In, Not Bolted On
One of the most valuable aspects of this system is how observability is woven into every layer from the start. Each service exposes its health status, and the API aggregates health checks across all dependencies. You can see at a glance whether the database is reachable, if the OCR service is responding, and if embeddings are processing correctly.
Real-Time Status Updates
The system uses Server-Sent Events (SSE) to push status updates from the backend to the frontend in real-time. When a document is uploaded, the user sees progress updates as it moves through the pipeline: uploaded → processing → OCR complete → embeddings generated → completed. Each status transition is broadcast to all connected clients, so multiple users can see updates simultaneously.
The worker sends webhook callbacks to the API at each stage, which then broadcasts via SSE. This creates a clear separation of concerns: workers focus on processing, the API handles coordination, and the frontend receives updates without polling.
Health Checks Everywhere
Every service exposes a /health endpoint. The API service aggregates these checks, allowing you to see the operational status of the entire system at once. Workers run periodic health checks on all their dependencies—database, Redis, MinIO, OCR service, embeddings service—logging the results so you can spot issues before they become problems.
The Bull Dashboard provides visibility into the job queue: how many jobs are waiting, which are processing, which completed successfully, and which failed. This queue visibility is crucial for understanding system load and identifying bottlenecks.
Worker Tracking and Cancellation
The Python services implement worker tracking to identify which thread is processing which request. This enables graceful cancellation—if a user cancels a document processing job, the system can stop the OCR or embedding operation mid-flight rather than wasting resources on work that's no longer needed.
Cancellation is handled at multiple levels: the queue supports job cancellation, the worker checks for cancellation before each step, and the Python services detect disconnections to abort long-running operations. This creates a responsive system where users aren't stuck waiting for work they've already cancelled.
Scalability Through Design
The system is designed for horizontal scalability from day one. You can scale the OCR service horizontally by running multiple containers—Docker Compose makes this trivial with docker-compose up --scale oc=3. Each OCR instance processes jobs independently, and the queue distributes work across all available workers.
The worker pool can also scale horizontally. Multiple worker instances pull jobs from the same Redis queue, processing documents in parallel. Concurrency is configurable per service—you can tune how many documents each worker processes simultaneously, and how many threads each OCR container uses.
This design means you can add capacity by adding more containers, not by buying bigger machines. Need to process more documents? Scale up the workers. OCR becoming a bottleneck? Scale up the OCR service. The architecture doesn't fight you—it's built for this.
Resilience and Graceful Degradation
The system is designed to work even when some services are down. If the OCR service is unavailable, jobs fail gracefully with clear error messages rather than hanging indefinitely. If the embeddings service is down, documents can still be processed through OCR—the system degrades functionality rather than crashing entirely.
File uploads use streaming to handle large files efficiently. The API streams files directly to MinIO storage rather than buffering everything in memory, allowing the system to handle many concurrent uploads without running out of memory.
Error handling is consistent across services. Failed jobs are retried with exponential backoff, and failures are logged with context so you can understand what went wrong. The database tracks document status through clear state transitions: pending → uploaded → processing → completed (or failed).
The Good Stuff: What Makes It Work
What I'm most proud of isn't any specific feature—it's how the pieces fit together:
Clear Service Boundaries: Each service has a single, well-defined responsibility. The API handles HTTP requests and coordination. Workers handle job processing. OCR extracts text. Embeddings generate vectors. This separation makes the system easier to understand, test, and modify.
Consistent Communication Patterns: Services communicate through well-defined interfaces. The API exposes REST endpoints with Swagger documentation. Python services expose FastAPI endpoints with automatic OpenAPI docs. Workers use webhooks to send status updates. These patterns are consistent and predictable.
Database Design for Search: The database uses PostgreSQL's full-text search (tsvector) alongside vector similarity search (pgvector). Documents can be found by keyword search or semantic similarity, and the system uses appropriate indexes (GIN for full-text, IVFFlat for vectors) to make both fast.
Real-Time Updates Without Complexity: SSE provides real-time updates without the complexity of WebSockets. The API maintains a simple set of connected clients and broadcasts updates when status changes. This is easier to reason about than bidirectional WebSocket connections.
Development Experience: The system uses Docker Compose for local development, with volume mounts for hot-reload. Code changes are reflected immediately without rebuilding containers. Each service has its own environment configuration, making it easy to test different scenarios.
Building for Scale
The real test of good architecture isn't how it performs with one user—it's how it performs with hundreds of concurrent users processing thousands of documents. This system is designed for that scale.
Jobs are queued rather than processed synchronously, so the API stays responsive even under heavy load. File storage uses object storage (MinIO) rather than the filesystem, so storage can scale independently. Database queries use appropriate indexes, and vector search uses specialized indexes for fast similarity queries.
The system can handle failures gracefully. If a worker crashes mid-processing, the job can be retried. If a Python service becomes unresponsive, health checks detect it and new requests fail fast rather than hanging. The queue provides durability—if Redis restarts, jobs are preserved.
Conclusion
Building scalable full-stack systems isn't about using the latest framework or the trendiest database. It's about thoughtful design, clear service boundaries, comprehensive observability, and resilience from the start.
The scalable-fullstack repository demonstrates these principles in practice. Each service is independent, observable, and scalable. The system handles failures gracefully, provides real-time updates, and can grow to handle increased load by adding more instances rather than rewriting code.
The best part? These principles aren't specific to OCR applications or microservices. They're universal ideas that apply whether you're building APIs, data pipelines, or any system that needs to handle scale and complexity. The tools change, but the fundamentals remain the same: clear boundaries, observability, resilience, and scalability through design.