rag
AI Research Workspace
Most of my earlier projects were built around Express.js for the backend. While working on AI-focused applications, I realized that building serious AI systems requires a different backend ecosystem and a deeper understanding of how AI workflows actually operate. That led me to learn FastAPI, and I used this project to not only learn it, but also polish my backend and AI engineering skills by building something from the ground up. Introducing 🚀 AI Research Workspace (AIRW) — an AI-powered workspace designed to interact with and retrieve information from different types of user data. 🔹 What I explored and implemented: • FastAPI for the backend and API architecture • LangChain & LangGraph for building structured AI workflows and agentic execution • RAG (Retrieval-Augmented Generation) for document-based question answering • Document chunking, embeddings, vector search & hybrid retrieval • Semantic retrieval using embeddings combined with traditional keyword-based retrieval • Query rewriting and retrieval pipelines to improve search quality • MongoDB Atlas Vector Search for storing and retrieving embeddings • Supabase Storage for handling uploaded files and generated content • Groq for LLM-powered generation and reasoning • Tavily for web-based research when web access is enabled • Memory system for retaining useful user-specific information • Tool-based agent execution with planning and execution stages 📚 Supported data & AI capabilities AIRW can work with multiple types of content, including: 📄 PDF documents 📊 CSV & Excel files 🎧 Audio files with transcription and timestamp-based retrieval 🖼️ Images using vision capabilities For documents, the system processes the content into smaller chunks, generates embeddings for those chunks, and stores them for semantic retrieval. When a user asks a question, AIRW retrieves the most relevant chunks. The system can also provide sources, allowing users to see where the information came from 🤖 It isn't limited to document Q&A either. AIRW supports: • Document summarization • Retrieval-based questions • General conversation • Web-assisted research • Memory-based responses • Python-powered analysis and visualization • Excel automation and modification • Multi-step tool execution through an agent workflow One of the things I particularly enjoyed was implementing many of these components myself instead of treating AI frameworks as black boxes. Understanding how chunking, embeddings, retrieval, planning, execution, memory, and tool orchestration work together has been one of the most valuable parts of this project. Github Repo: https://github.com/WaleedImran2007/AI-Research-Workspace Live Demo: https://airw.waleedimran.me Still learning. Still improving. 🚀 AI ArtificialIntelligence AIEngineering GenerativeAI RAG LangChain LangGraph FastAPI Python MongoDB AgenticAI MachineLearning BackendDevelopment FullStackDevelopment
🚀 Built a Basic RAG and an Advanced RAG System from Scratch — Without LangChain
To understand how RAG works internally, I first built a Basic RAG Pipeline: Basic RAG PDF → Extraction → Chunking → Embeddings → FAISS Search → Top-K Chunks → Prompt → LLM → Answer It works well for simple questions, but struggles with exact keywords, IDs, and complex queries. So I built an Advanced RAG Pipeline: Advanced RAG PDF → Extraction → Chunking → Embeddings → Dense + Sparse Index → Query Understanding → Dense Retrieval + Sparse Retrieval → Reciprocal Rank Fusion (RRF) → Cross-Encoder Reranking → Noise Filtering → Context Fusion → Prompt → LLM → Answer Key Improvements ✅ Semantic Search (FAISS) ✅ Exact Keyword Search (BM25) ✅ Hybrid Retrieval ✅ Cross-Encoder Reranking ✅ Duplicate Removal ✅ Better Context Selection ✅ Reduced Hallucinations One of the biggest lessons from this project: A better RAG system isn't always about using a better LLM. It's often about building a better retrieval pipeline. Tech Stack: Python • Streamlit • FAISS • Sentence Transformers • BM25 • Cross-Encoder Reranker • Groq RAG GenAI LLM AIEngineering MachineLearning Python FAISS BuildInPublic
Shipped: Corvit AI Chatbot - Smarter Way to Help Students
What if students could just ask instead of searching 5 different campus pages? I built an AI-powered chatbot for Corvit Systems that answers questions about: • Courses, certifications, and NAVTTC programs • Class timings, instructors, and facilities • Fees, career opportunities, and more Key Features: AI-powered conversational assistance RAG-based knowledge retrieval for accurate answers Context-aware responses + online search Clean web interface Tech Stack: GPT-OSS-120B | RAG | JavaScript | Tailwind CSS | Netlify Goal: Turn a traditional info website into an intelligent student assistant. More improvements coming soon! AI RAG Chatbot WebDev JavaScript EdTech CorvitSystems