Project Overview
MandarinClips is a high‑performance language learning platform that indexes sentence‑sized dialogue segments from authentic Chinese movies and TV series. By presenting real‑world conversational snippets, the service helps learners move beyond textbook examples and engage with natural language usage.
Core Architecture
The system is designed for speed, cost efficiency, and high availability. The technology stack consists of:
- Frontend: Next.js 15 with React 19, Tailwind CSS, and next‑intl for multilingual support.
- Backend: Rust built on the Axum framework, running on the Tokio asynchronous runtime.
- Database: PostgreSQL enhanced by the
pg_trgmextension and GIN indexes for fuzzy text search. - Storage & CDN: Backblaze B2 storage accessed via the S3 API and delivered through Cloudflare CDN, providing zero‑cost egress and global caching.
Why Rust and Axum Were Chosen
The backend handles high‑frequency search queries, sitemap generation, and user quota tracking. Rust offers:
- Memory Safety and Zero‑Cost Abstractions – Absence of garbage collection eliminates pause times, ensuring predictable latency.
- Axum + Tokio – Axum provides a lightweight routing layer while Tokio manages asynchronous tasks with minimal CPU overhead. A single 2‑core virtual private server can sustain thousands of concurrent requests without degradation.
Sub‑Millisecond Fuzzy Search with PostgreSQL
With more than 100,000 video clips and associated subtitles, a naïve ILIKE '%query%' approach would trigger full‑table scans and overload the CPU. The solution leverages the pg_trgm extension to generate trigram‑based GIN indexes on both Chinese characters and tone‑less pinyin strings. This configuration enables rapid similarity searches while keeping CPU usage low.
“GIN indexes combined with trigram matching deliver sub‑millisecond response times even under heavy load.”
Example index creation:
CREATE EXTENSION IF NOT EXISTS pg_trgm;
CREATE INDEX idx_clips_text_zh_trgm ON clips USING GIN (text_zh trigram_ops);
CREATE INDEX idx_clips_text_pinyin_trgm ON clips USING GIN (text_pinyin trigram_ops);
Content Delivery and Cost Management
Video assets are stored in Backblaze B2, an affordable object storage service compatible with the S3 API. Cloudflare’s CDN proxies these files, providing edge caching, DDoS protection, and zero egress fees for most traffic patterns. This architecture reduces operational costs while maintaining fast load times for global users.
Frontend Performance Strategies
Next.js 15 enables server‑side rendering (SSR) and incremental static regeneration (ISR), which pre‑render frequently accessed pages and reduce client‑side computation. Tailwind CSS provides utility‑first styling that compiles to minimal CSS bundles, improving page load speed. The next‑intl library supplies locale‑aware formatting, allowing learners to switch seamlessly between Chinese characters, pinyin, and English translations.
Scalability and Reliability
All components are containerizable and can be orchestrated with Kubernetes or simple Docker Compose deployments. Horizontal scaling of the Rust backend is straightforward because each request is handled asynchronously with Tokio, minimizing thread contention. PostgreSQL can be scaled vertically or replicated using read‑only followers to balance search load.
Key Takeaways
- Rust and Axum deliver low‑latency, memory‑safe backends suitable for intensive search workloads.
- Trigram‑based GIN indexes in PostgreSQL provide fast fuzzy matching for Chinese and pinyin text.
- Combining Backblaze B2 with Cloudflare CDN ensures cost‑effective, globally distributed media delivery.
- Next.js 15 with SSR and ISR optimizes frontend rendering for a smooth user experience.
Future Enhancements
Planned improvements include adding AI‑driven subtitle alignment, expanding the video library beyond 100k clips, and integrating user analytics to personalize learning pathways. Continuous monitoring with Prometheus and Grafana will guide performance tuning as traffic grows.
Leave a Reply