RAG

For a while now, I've been experimenting with Retrieval-Augmented Generation (RAG), exploring how to make LLMs answer questions using information retrieved from actual documents. Today's experiment with Raggy took me through another complete RAG pipeline. Document ingestion: Processed a research paper in PDF format. Text chunking: Split the document into 82 chunks. Vector embeddings: Used nomic-embed-text to convert text into 768-dimensional vector representations. Semantic search: Retrieved the top 5 chunks based on similarity to the user's query. LLM inference: Used a locally running qwen3:4b model to generate an answer from the retrieved context. Retrieval tracing: Inspected chunk rankings, similarity scores, source pages, token usage, and execution timings. Then I asked: “What was the IoT adoption rate across the companies?” Raggy returned 74.4%, grounded in the retrieved document context. But here's where it gets interesting. The trace exposed duplicate chunks in the retrieval results. The generation stage also took significantly longer than embedding and retrieval in this run. These are exactly the kinds of details I want to investigate. Because building RAG isn't simply about connecting a vector database to an LLM. It's about understanding document parsing, chunking strategies, embedding models, vector similarity, top-k retrieval, context construction, grounding, and inference latency. A system can return the right answer and still have room for improvement. My next focus is making retrieval more reliable, reducing redundant context, and improving response time without sacrificing answer quality. Still experimenting. Still debugging. Still learning by building. The goal isn't just to make AI work. It's to understand the engineering behind making it work well. #RAG #AIEngineering #LLM #Python #MachineLearning #softwareengineering

1 / 3

Add a comment

0/2000