Chat Streaming (SSE)

The chat system uses Server-Sent Events (SSE) for real-time streaming of AI responses, creating the “typing” effect users expect.

Why SSE over WebSocket?

Feature	SSE	WebSocket
Direction	Server → Client	Bidirectional
Complexity	Simple HTTP	Requires upgrade handshake
Reconnection	Automatic	Manual

For AI chat, we only need server → client. SSE is simpler and works perfectly.

Event Types

Event	When	Data
message_start	Stream begins	{ conversation_id, message_id }
message_delta	Each token chunk	{ content: "token text" }
message_end	Stream complete	{ tokens_used, cost_usd, citations }
error	On failure	{ code, message }

Data Flow

1. User sends message → optimistic update (message appears immediately)
2. Frontend POSTs to /api/v1/chat/stream with Accept: text/event-stream
3. Backend creates StreamingResponse with async generator
4. NutritionChatAgent builds RAG context → streams tokens from LLM
5. Frontend accumulates tokens via useSSE hook → renders in real-time
6. On complete → full message saved to state with citations and metadata

Key Files

Frontend

File	Purpose
hooks/useChat.ts	All chat state management
hooks/useSSE.ts	SSE streaming state
api/chatClient.ts	streamMessage() with SSE parsing
components/chat/ChatInput.tsx	Fixed input at viewport bottom

Backend

File	Purpose
api/v1/chat.py	SSE endpoint + conversation CRUD
agents/nutrition_chat.py	AI agent with streaming
services/context_builder.py	RAG context assembly

Features

• Optimistic updates — user messages appear before API response
• Abort streaming — cancel mid-response via AbortController
• Citations — knowledge sources returned in message_end
• Meal references — referenced meals rendered as mini-cards
• Fixed input — chat input stays at viewport bottom (position: fixed)