Flagship Project

MHT-ChatRAG

A real-time, low-latency, multilingual AI assistant powered by a complex Retrieval-Augmented Generation (RAG) pipeline and a robust Tri-DB persistence layer.

Python LangChain RAG (Hybrid) LLMs (GPT/LLaMA/Gemini) Prompt Engineering APIs REST Node.js WebSockets Redis PostgreSQL LanceDB ElevenLabs TTS AssemblyAI (STT) Docker Live2D Cubism

System Architecture

┌─────────────────────┐ ┌─────────────────────┐ │ Browser / UI │ HTTP/WS (PORT 3000) │ Express Server │ │ - clientChatHandler│◄────────────────────────────►│ - WebSocket Server │ │ - Live2D Avatar │ │ - REST API │ │ - STT/TTS Client │ │ - Static Files │ └─────────────────────┘ └──────────┬──────────┘ │ ┌──────────────────────────────────────────┼──────────────────────┐ │ │ │ ▼ ▼ ▼ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ Redis │ │ PostgreSQL │ │ LanceDB │ │ - Session Cache │ │ - Chat History │ │ - Vector Store │ │ - Message Queue │ │ - Payloads │ │ - Embeddings │ └─────────────────┘ └─────────────────┘ └─────────────────┘

Tri-DB Resilience

Redis manages ultra-fast session cache and queues. PostgreSQL acts as the structured backbone for history and variables. LanceDB holds the permanent vector archive. Disconnections trigger instant restoration from Postgres to prevent LLM context loss.

Last-Write-Wins Concurrency

Handles barge-in natively. If a new message arrives while the LLM is generating, the obsolete stream is proactively aborted (`break`), freeing resources and focusing on the new interaction.

Live2D Avatar Sincronization

Integrating a Live2D model (Hiyori) via an iframe with a bidirectional `postMessage` queue. Combines Web Audio API and TTS streaming to achieve real-time lip-sync animations with no noticeable delay.

Security & Modularity

Built with a dedicated JS obfuscation pipeline to protect client-side RAG logic. Employs dynamic component injection for UI elements like modales, avoiding monolithic architectures.

Multimedia Showcase

See the real-time interaction, Live2D synchronization, and RAG pipeline in action.

AI Chat Assistant with Live2D Avatar — Real-time voice & text interaction

Demo: Full E2E interaction — STT → RAG → LLM Streaming → TTS → Lip-Sync

RAG Pipeline Architecture - Hybrid retrieval with multi-provider LLM fallback

RAG Pipeline Architecture — Embedding → Vector Search → Multi-Provider LLM → TTS

Advanced RAG Pipeline

The system combines Vector Search and Full-Text Search (FTS) for a Hybrid Retrieval approach. It adapts embeddings on-the-fly depending on the context and dynamically orchestrates chains.

// Simplified Pipeline Flow
Query → Hybrid Retriever → Vector + FTS Results
                                      ↓
                          Context Assembly
                                      ↓
                          LLM Generation (Cerebras/Gemini/OpenAI/Ollama)
                                      ↓
                          Response Stream (TTS Chunking)
                

Key UX Innovations

Audio Streaming Bidirectional: Reproduces chunks from Edge-TTS while the LLM generates the rest, drastically reducing Time-To-First-Byte (TTFB).
Timezone-Awareness (`dateUtils.js`): Parses temporal references ("today", "tomorrow") relative to the client's local timezone (UTC-5), intelligently handling server midnight rollovers.
Transient Messages & Debounce: Shows localized "I couldn't hear you" messages on STT failure without polluting the conversation history, paired with a 500ms microphone debounce to prevent accidental double-clicks.