ARCH — Chat With Books

Split ingestion into two phases instead of one long pipeline: parse the PDF/EPUB into chapters (TOC first, font-size heuristics as fallback), chunk each chapter into overlapping 800-token pieces, and embed them locally with all-MiniLM-L6-v2 into Postgres/pgvector. That's Phase 1, and it's the only thing gating chat — a book is usable within seconds instead of waiting on the slower map-reduce summarization pass, which runs as a best-effort background job (Phase 2) that can fail per-chapter without ever blocking the chat experience.

Built retrieval as a plain cosine search over pgvector with an empirically-calibrated distance threshold (0.82, tuned to this embedding model's own geometry rather than borrowed from elsewhere), plus a chapter-number fallback so a question like 'what happens in chapter 3' still works even if nothing clears the similarity bar. Answers stream token-by-token over SSE from Groq, with every response required to cite the chapter and page range it came from — skipped entirely if no chunk is actually relevant, rather than letting the model improvise.

Kept the whole thing dependency-light on purpose: no LangChain or LlamaIndex between the code and the pipeline, so every step — parsing, chunking, retrieval, map-reduce summarization — is code I can read end to end. Conversation history lives entirely server-side (Postgres), the frontend never re-sends it, and a three-state sentinel (NULL/empty-string/value) on summary fields lets the UI distinguish 'still generating' from 'failed' from 'done' without an extra status column.