Been wanting to build this for a while, finally sat down and did it. It’s a Flask app where you upload a PDF, it chunks and embeds it, and then you can ask questions and get answers pulled only from that document, not from the model’s own training data. Stack is pretty simple: Ollama for the chat model and the embedding model, ChromaDB as the vector store, Flask tying it together. Nothing exotic. How it works, roughly: PDF gets split into overlapping chunks so sentences don’t get cut off between pieces Each chunk gets turned into an embedding and stored in Chroma with PersistentClient, so it’s saved on disk instead of disappearing every time you restart the app When you ask something, the question also gets embedded, Chroma finds the closest matching chunks, and those get handed to the model as context Prompt explicitly tells the model to only use that context and say it doesn’t know if the answer isn’t there, otherwise it’ll just make something up from its own memory Tested it by asking something not in the PDF and it correctly said it didn’t know instead of guessing. Also tested with wifi off and it kept working, since the model, embeddings, and vector store all run locally with no external api calls in the loop. submitted by /u/SilverConsistent9222
Originally posted by u/SilverConsistent9222 on r/ArtificialInteligence