大模型
dev.to ·
@David Díaz
UAJY 手册 RAG 聊天机器人：FAISS 检索与 Gemini 生成分离
公开仓库 UAJY Academic Document RAG Chatbot 为 Universitas Atma Jaya Yogyakarta 工业技术学院 2025/2026 学年手册提供 Streamlit 助手。离线流程用 pdfplumber 抽取文本与表格，按语义或段落切块，用 gemini-embedding-001 生成 3072 维向量并存入 FAISS 索引（含页码与标题元数据），仓库称索引含 112 页手册的 350 个 chunk。运行时 Streamlit 查询触发 FAISS top-K 相似检索，将检索上下文与对话历史拼成提示词发送给 Gemini 2.5 Flash。代码库还包含引用生成、拒答控制和 20 题评测脚本。
AI推荐理由 提供可检视的 RAG 实现细节：pdfplumber 抽取文本与表格、gemini-embedding-001 生成 3072 维向量、FAISS 索引 350 个 chunk（来自 112 页手册）、Streamlit 前端、Gemini 2.5 Flash 生成带引用回答，并含拒答控制与 20 题评测集。属于具体工程实现分享，但为个人项目，未提供评测结果或效果数据，价值主要在参考架构而非突破。
62
正文
The public UAJY Academic Document RAG Chatbot repository packages a Streamlit assistant for the 2025/2026 academic handbook of Universitas Atma Jaya Yogyakarta’s Faculty of Industrial Technology. For developers, it provides an inspectable implementation of document ingestion, local vector search, cited answer generation, refusal controls and a 20-question evaluation harness in one codebase.
From handbook PDF to cited answer
The offline ingestion pipeline extracts text and tables with pdfplumber, divides the material into semantic or paragraph-based chunks, creates 3,072-dimensional embeddings with gemini-embedding-001, and persists the vectors in a FAISS index alongside page and heading metadata. The repository says the resulting index contains 350 chunks drawn from 112 handbook pages.
At runtime, a Streamlit query triggers top-K similarity retrieval from FAISS. The application builds a prompt from the retrieved context and conversation history, sends it to Gemini 2.5 Flash, and formats the answer with page numbers and chapter or section titles, according to the documented architecture.
The project describes two controls for unsupported questions: a similarity threshold filters retrieval results, while the system prompt instructs Gemini to refuse requests that the retrieved document context cannot answer. The README characterizes these controls as an anti-hallucination defense and says generated answers must rely exclusively on retrieved PDF chunks.
What the published evaluation covers
The repository includes an evaluation runner and a test file with 20 questions. Its benchmark summary reports Retrieval Recall@4 of 100% for 15 in-scope questions, refusal correctness of 100% for five out-of-scope questions, average retrieval latency of 0.42 seconds and total response latency of approximately 1.85 seconds. These figures describe the project’s published test suite.
Analysis: an inspectable design with bounded evidence
Analysis: The repository is best read as a concrete RAG implementation whose grounding mechanisms can be inspected, rather than conclusive evidence for its broader “production-grade” and “hallucination-free” labels. The published summary establishes reported behavior for 20 prompts; it does not, by itself, establish behavior beyond that suite. The project’s own disclaimer draws a similar boundary by describing the chatbot as an educational and information-search assistant while reserving official authority for the deanery and academic administration office. Project README
The unresolved deployment trade-off is the boundary between local and hosted components. The FAISS index and CPU-based similarity search run locally, while embedding creation and answer generation use Google Gemini services through an API; setup therefore requires a Gemini API key. Repository architecture and setup Analysis: This gives adopters a local retrieval layer, but not a fully local RAG runtime. The relevant engineering decision is whether that split fits the intended deployment environment—not whether local vector search alone is sufficient.