Flagship Project

MHT-ChatRAG

A real-time, low-latency, multilingual AI assistant powered by a complex Retrieval-Augmented Generation (RAG) pipeline and a robust Tri-DB persistence layer.

Node.js WebSockets Redis PostgreSQL LanceDB AssemblyAI (STT) Edge-TTS Live2D Cubism

System Architecture

┌─────────────────────┐ ┌─────────────────────┐ │ Browser / UI │ HTTP/WS (PORT 3000) │ Express Server │ │ - clientChatHandler│◄────────────────────────────►│ - WebSocket Server │ │ - Live2D Avatar │ │ - REST API │ │ - STT/TTS Client │ │ - Static Files │ └─────────────────────┘ └──────────┬──────────┘ │ ┌──────────────────────────────────────────┼──────────────────────┐ │ │ │ ▼ ▼ ▼ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ Redis │ │ PostgreSQL │ │ LanceDB │ │ - Session Cache │ │ - Chat History │ │ - Vector Store │ │ - Message Queue │ │ - Payloads │ │ - Embeddings │ └─────────────────┘ └─────────────────┘ └─────────────────┘

Tri-DB Resilience

Redis manages ultra-fast session cache and queues. PostgreSQL acts as the structured backbone for history and variables. LanceDB holds the permanent vector archive. Disconnections trigger instant restoration from Postgres to prevent LLM context loss.

Last-Write-Wins Concurrency

Handles barge-in natively. If a new message arrives while the LLM is generating, the obsolete stream is proactively aborted (`break`), freeing resources and focusing on the new interaction.

Live2D Avatar Sincronization

Integrating a Live2D model (Hiyori) via an iframe with a bidirectional `postMessage` queue. Combines Web Audio API and TTS streaming to achieve real-time lip-sync animations with no noticeable delay.

Security & Modularity

Built with a dedicated JS obfuscation pipeline to protect client-side RAG logic. Employs dynamic component injection for UI elements like modales, avoiding monolithic architectures.

Multimedia Showcase

Watch the real-time interaction and Live2D synchronization in action.

[ Video Component / Gallery will be inserted here ]
Showcasing UI, Live2D avatar, and Real-Time TTS/STT speeds.

Advanced RAG Pipeline

The system combines Vector Search and Full-Text Search (FTS) for a Hybrid Retrieval approach. It adapts embeddings on-the-fly depending on the context and dynamically orchestrates chains.

// Simplified Pipeline Flow Query → Hybrid Retriever → Vector + FTS Results ↓ Context Assembly ↓ LLM Generation (Cerebras/Groq/OpenAI) ↓ Response Stream (TTS Chunking)

Key UX Innovations

  • Audio Streaming Bidirectional: Reproduces chunks from Edge-TTS while the LLM generates the rest, drastically reducing Time-To-First-Byte (TTFB).
  • Timezone-Awareness (`dateUtils.js`): Parses temporal references ("today", "tomorrow") relative to the client's local timezone (UTC-5), intelligently handling server midnight rollovers.
  • Transient Messages & Debounce: Shows localized "I couldn't hear you" messages on STT failure without polluting the conversation history, paired with a 500ms microphone debounce to prevent accidental double-clicks.